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Abstract 


Change  implies  risk.  There  are  elements  of  risk  in  all  systems 
integration  projects.  Sometimes  the  risk  is  negligible,  sometimes  it 
is  great  but  it  must  always  be  identified,  measured  and  minimized. 
Many  service  vendors  and  their  customers  utilize  their  contract  as  a 
mechanism  for  reducing  their  exposure  to  risks.  This  report 
examines  how  risk  reduction  is  approached  from  the  vendor's  and 
customer's  points  of  view. 

The  report  identifies  the  elements  of  the  work  that  are  perceived  as 
having  an  element  of  risk.  Customers  do  not  always  share  the 
vendors  view  of  where  risk  may  exist  or  the  extent  to  which  part  of 
the  project  may  have  high  risk.  These  areas  of  difference  are 
highlighted. 

Vendors  use  differing  techniques  to  determine  the  potential  risk 
within  a  project.  Their  analysis  is  often  a  key  determinant  of  the 
proposed  project  price.  As  the  systems  integration  market  changes 
and  the  types  of  projects  evolve,  then  pricing  structures  also  evolve. 
This  report  ties  together  some  of  the  commonly  used  pricing 
practices  and  the  associated  project  types. 

Once  the  price  and  contract  are  in  agreement,  it  is  then  necessary 
to  manage  the  project  and  the  associated  risk.  There  are  a  variety 
of  tools  and  practices  being  used  in  this  area.  These  are  discussed 
and  trends  are  identified. 

The  report  concludes  with  an  examination  of  the  relationship 
between  users  and  vendors  and  a  simimary  of  trends  in  risk 
management  within  user  organizations  as  well  as  within  vendor 
companies. 

This  report  contains  86  pages  and  25  exhibits. 
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Introduction 


Undertaking  any  significant  systems  integration  (SI)  project, 
whether  in-house  or  in  conjunction  with  a  professional  services  (PS) 
or  SI  vendor,  usually  entails  some  significant  risks.  These  projects 
tend  to  be  large,  frequently  involve  the  use  of  new  technologies  and 
require  sophisticated  management  processes  to  ensure  that: 

•  The  delivered  functionality  meets  user  requirements. 

•  Schedules  and  deadlines  are,  in  fact,  met. 

•  Actual  costs  are  held  close  to  forecasted  costs. 

•  Implementation  of  the  resulting  systems  and  related  business 
processes  are  accomplished  in  an  orderly  and  effective  manner. 

The  history  of  major  systems  development  efforts  contains 
numerous  examples  of  projects  that  did  not  meet  at  least  one  of 
these  objectives,  and  there  is  no  shortage  of  examples  that  failed  on 
two  or  more.  To  some  degree  the  very  existence  and  rapid  growth 
in  the  use  of  systems  integrators  over  the  past  decade  has  been 
fueled  by  the  user  community's  desire  to  reduce  the  risks  and 
increase  the  probability  of  success  for  major  SI  efforts  by  partnering 
with  firms  who  specialize  in  the  development  and  implementation 
of  complex  information  systems. 

Users  look  to  these  firms  to  provide: 

•  Expertise  in  the  applications  relevant  to  their  industry 

•  Experience  in  the  use  of  current  technology 
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•  Sophisticated  tools  and  processes  to  insure  accurate  and  timely 
implementation 

It  is  safe  to  assume  that  all  of  these  things  contribute  to  reducing 
the  risks.  Nevertheless,  risks  still  and  will  continue  to  exist  in 
major  systems  integration  projects,  with  the  sources  of  risk 
frequently  perceived  differently  by  SI  vendors  and  users  of  their 
services. 

A  

Purpose 

The  purpose  of  this  study  is  to  examine  how  the  risks  are  perceived 
by  both  parties,  and  gain  an  understanding  of  what  mechanisms, 
contractual  and  otherwise,  are  put  into  place  to  contain  risk  in  SI 
engagements.  Specifically,  this  study  identifies  user  and  vendors 
views  on: 

•  The  areas  of  risk  associated  with  SI  projects  and  their  impact 
on  the  project  outcome 

•  Approaches  to  assessing  risks  for  specific  projects 

•  Vendor  and  user  attitudes  toward  the  sharing  of  risks 

•  The  contract  mechanisms  and  management  processes  used  to 
control  risks 

•  The  impact  of  risk  on  project  pricing 


B  

Scope  and  Use 

The  scope  of  this  study  is  limited  to  users  and  SI  vendors  in  the 
United  States. 

From  a  vendor's  perspective  the  study  should  prove  useful  by: 

•    Providing  insight  into  how  users  see  the  risks  associated 
with  SI  contracts 
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•  Identifying  the  way  other  vendors  deal  with  risk-related 
issues  contractually 

•  Pinpointing  tools  and  methodologies  employed  to  manage 
risk 

Users  should  also  benefit  from  the  findings,  by  gaining  a  greater 
understanding  of  the  wide  variety  of  approaches  taken  by  various 
vendors  to  risk  assessment  and  management  and  the  impact  of  risk 
on  vendor  pricing. 


Methodology/Demographics 

1.  Methodology 

To  obtain  the  data  used  in  this  analysis,  interviews  were  conducted 
with  65  companies  who  have  used  systems  integrators  in  a  least 
one  engagement.  In  addition,  11  interviews  were  conducted  with 
major  systems  integrators.  (Appendices  B  and  C  contain  the  user 
and  vendor  interview  guides). 

In  some  instances  users  and  vendors  were  asked  identical  questions 
to  allow  for  a  direct  comparison  of  the  responses. 

In  addition  to  the  data  gathered  through  the  field  interviews, 
information  from  INPUT'S  prior  systems  integration  research, 
along  with  data  from  secondary  sources,  was  used  to  formulate  the 
conclusions  and  recommendations  presented  here. 

2.  Demographics 

In  general,  the  user  firms  participating  m  the  study  were  large. 
The  average  annual  sales  volume  for  the  65  companies  was  just 
over  $3  billion.  The  average  number  of  employees  was  10,600. 

As  shown  in  Exhibit  I-l,  a  wide  variety  of  industries  was 
represented. 
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Exhibit  1-1 


Distribution  of  User  Respondents  by  Industry 


Industry  Group 


lanufacturing 
Government 
Distribution 
Finance 
Utilities 
Telecom 
Education 
Business  Svcs 
Other 


0  2  4  6 

Number  of  Respondents 


10 


12 


Eighty  five  percent  (85%)  of  the  user  respondents  were  CIOs  or  the 
heads  of  information  systems  from  major  divisions  of  large 
corporations.  The  remaining  15%  consisted  of  general  managers  or 
contract  specialists. 

On  the  vendor  side  the  variety  was  somewhat  greater.  In  the  case 
of  the  Big  6  companies,  partner-level  individuals  with  direct 
practice  experience  were  the  rule.  For  hardware  companies, 
respondents  tended  to  be  individuals  responsible  for  contract 
strategy  for  the  organization's  professional  services,  or  systems 
integration  division. 

In  some  instances  more  than  one  individual  was  contacted  within  a 
user  or  vendor  firm  to  obtain  the  necessary  information.  In  those 
cases  a  single  questionnaire  was  prepared  reflecting  the  composite 
view. 
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Organization 


The  remainder  of  the  report  is  organized  into  four  chapters: 

•  Chapter  II,  Executive  Overview — ^provides  a  synopsis  of  the 
findings  of  this  study  along  with  recommendations. 

•  Chapter  III,  Users'  Perspectives  on  Risk — ^presents  INPUT'S 
analysis  of  how  users  perceive  risk  when  undertaking  an  SI 
project  with  an  outside  vendor,  the  processes  they  deploy  to 
control  the  elements  of  risk,  and  the  types  of  contract 
mechanisms  they  feel  are  effective  in  managing  the  risk. 

•  Chapter  IV,  Vendors'  Perspectives  on  Risk — examines  the  same 
issues  as  Chapter  III,  but  from  a  vendor  perspective  and 
identifies  areas  where  vendors  and  users  differ  in  their  views 
regarding  risk  and  its  management. 

•  Chapter  V,  Risk  in  Contracts—discusses  user/vendor 
relationships  and  identifies  current  trends  in  SI  contracts 

•  Appendix  A,  Definition  of  Terms — ^provides  definitions  of  key 
terms  used  throughout  this  report.  A  complete  set  of  INPUT 
industry  definitions  may  be  obtained  by  contacting  any  INPUT 
office. 

•  Appendices  B  and  C  contain  the  user  and  vendor  interview 
guides  used  for  this  study. 


Related  Reports 

U.S.  Systems  Integration  Market  Forecast,  1993-1998 
Systems  Integration  Opportunities  in  Reengineering 
Systems  Integration  in  Discrete  and  Process  Manufacturing 
Systems  Integration  in  Health  Services 
Systems  Integration  in  State  and  Loccd  Government 
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Executive  Overview 


Over  the  past  ten  years,  users  have  increasingly  used  professional 
services  and  systems  integration  firms  to  assist  them  in  major 
systems  development  efforts.  They  look  to  these  vendors  for 
application  and  industry  expertise,  currency  in  new  technology,  and 
overall  management  of  the  implementation.  Furthermore,  since 
systems  integration  (SI)  agreements  assign  primary  responsibility 
for  a  project  delivery  to  the  vendor,  users  anticipate  that  the  vendor 
will  assume  the  bulk  of  the  responsibility  for  managing  risk.  Yet, 
despite  the  growing  sophistication  in  tools  and  methodologies, 
undertaking  a  major  systems  integration  effort  still  involves 
significant  elements  of  risk  to  both  parties. 

How  these  risks  arc  perceived,  evaluated  and  managed  impacts  the 
nature  of  the  contractual  agreement  between  the  parties  and  the 
working  relationship  between  the  vendor  and  buyer. 

The  objective  of  this  study  is  to: 

•  Gain  insight  into  how  both  parties  perceive  and  evaluate  risk 

•  Provide  an  understanding  of  how  risk  is  managed  in  SI 
contracts 

•  Examine  the  processes  used  to  minimize  and  manage  risk 
during  SI  engagements 

•  Identify  trends  in  risk  assessment  and  management  that  are 
likely  to  have  an  impact  on  future  contracting  strategies  and 
user/vendor  working  relationships. 
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Introduction 


To  gather  information  for  the  study,  INPUT  conducted  interviews 
with  11  systems  integrators  and  65  companies  that  had  used 
systems  integrators  within  the  past  two  years.  They  were  asked  to 
comment  on: 

•  Their  perceptions  on  the  sources  of  risk  and  the  impact  of 
each 

•  How  they  assessed  risk 

•  Contract  mechanisms  and  management  processes  used  to 
control  risk 

•  The  impact  of  risk  on  project  pricing 

•  Attitudes  toward  risk  sharing  with  vendors 

•  Their  views  on  the  quahty  of  the  relationship  with  their  SI 
vendor 

This  research  was  the  primary  source  of  information  used  in  the 
analysis  presented  here. 


B 


Sources  of  Risk 


Users  and  vendors  identified  10  key  areas  of  risk  associated  with 
systems  integration  efforts. 

•  The  initial  identification  of  requirements  by  the  user 

•  Lack  of  user  involvement  during  the  course  of  the  project 

•  Project  management  by  the  SI  vendor 

•  Project  management  by  user  personnel 

•  Resource  estimating  by  the  vendor 

•  Resource  estimating  by  the  user 
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•  Risk  evaluation  by  the  vendor  at  the  start  of  the  project 

•  Risk  evaluation  by  users  at  the  start  of  the  project 

•  Control  over  user  requests  for  change 

•  Complexity  of  the  technology 

In  general,  there  was  agreement  between  users  and  vendors  on  the 
potential  impact  of  each  of  these  sources  of  risk.  But  there  were 
some  differences  worthy  of  comment.  Exhibit  II- 1  shows  how  users 
and  vendors  compared  in  their  assessments  of  risk  levels. 


EXHIBIT  11-1 


Comparison:  User  and  Vendor  Assessments  of  Risk  Impact 


Source  of  Risk 


Requirements  Identification  by  User 
User  Involvement  During  the  Project 
Project  Management  by  User 
Resource  Estimating  by  User 
Risk  Assessment  by  User 
Resource  Estimating  by  User 
Project  Management  by  Vendor 
Risk  Assessment  by  Vendor 
Change  Control 
Complexity  of  the  Technology 


=  High 
=  Medium 
Low 


Impact  Assessment  by: 


User 


Vendor 
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The  three  major  areas  of  difference  are: 

•  Complexity  of  the  Technology — ^Vendors  see  the  introduction  of 
chent/server  (C/S)  and  other  new  technologies  as  much  riskier 
than  users.  To  some  degree,  users'  relative  lack  of  concern  is 
attributable  to  the  hype  about  the  ease  of  implementation. 
But  vendors  also  contribute  to  this  high  degree  of  comfort  that 
users  have  by  emphasizing  their  processes  for  establishing 
requirements,  prototyping  and  design,  leaving  the  user  to 
assume  that  once  design  is  completed,  technology  risks  are 
minimal. 

•  User-Risk  Assessment — Users  generally  consider  their  own 
assessment  of  the  risk  as  a  source  of  risk  itself.  But,  vendors 
disagree,  largely  because  in  the  end,  it  is  their  own  assessment 
of  the  risk  that  will  dominate  the  estimating  and  pricing  of  a 
given  engagement. 

•  User-Project  Management — Since  users  expect  vendors  to 
assume  the  responsibility  for  delivery,  they  seldom  see  the  role 
of  their  project  manager  or  liaison  person  as  a  high  source  of 
risk.  Vendors  take  the  opposite  view  indicating  that  the  user 
project  manager  is  the  key  player  on  the  user  side.  They  look 
to  this  individual  as  a  key  facilitator  and  a  gateway  to  other 
user  resources. 

But  despite  these  differences,  vendors  and  users  alike  agree  that  all 
the  sources  cited  above  had  potential  to  introduce  some  level  of  risk 
into  the  effort. 

c  ^  

Risk  Control  Strategies 

1.  The  User  View 

The  findings  indicate  that  while  users  recognize  the  elements  of 
risk  in  an  SI  project,  they  generally  feel  that  their  ability  to  have  a 
significant  impact  on  risk  containment  is  limited.  The  key  areas 
where  they  feel  they  can  make  a  contribution  are: 

•  Strong  compliance  with  change  control  procedures 

•  The  provision  of  dedicated  user  personnel  to  the  project  team 
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•  The  adoption  of  the  integrator's  project  management,  risk 
containment  and  other  methodologies 

Essentially,  users  hope  to  cover  their  risk  through  contract 
mechanisms.  They  have  a  strong  preference  for  fixed-price 
agreements,  rating  this  approach  above  all  others  as  shown  in 
Exhibit  11-2. 


EXHIBIT  11-2 

User  Preferences  for  Pricing  Mechanisms 

Mechanism 


Fixed-Price 


Time-and- 
Materials 


Value-based 


1.0  2.0  3.0  4.0  5.0 

Low  High 
User  Preference  Rating 


In  addition  to  limiting  their  risk  financially  through  the  fixed-price 
agreement,  users  negotiate  for  other  contract  mechanisms  to  limit 
their  risk. 

•  Inclusion  of  task  milestone  commitments  in  contracts  was 
indicated  as  a  key  strategy  of  94%  of  the  users  in  the  survey. 

•  Performance  clauses  and  warranties  were  cited  in  more  than 
75%  of  the  cases. 

•  Bonus  payments  are  also  used,  but  presently  by  only  20%  of 
the  respondents. 

Exhibit  II-3  shows  users  are  generally  satisfied  with  the  levels  of 
risk  containment  they  are  able  to  achieve  using  these  mechanisms. 
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Exhibit  11-3 

User  Levels  of  Satisfaction  with  Risk  Containment 


Low  High 
14%  17% 


Average 
69% 

Proportion  of  Users 

No.  of  Respondents  =  65 


2.  The  Vendor  View 

Vendors  on  the  other  hand,  assume  most  of  the  risk  and  have 
developed  many  sophisticated  apj^roaches  to  assessing  and 
managing  the  risk  associated  with  an  SI  engagement. 

Of  primary  importance  is  risk  assessment  itself.  Generally, 
vendors  approach  this  process  usnig  one  of  the  following  three 
approaches. 

•  Risk  Scojing-~Th.c  use  of  formal  scoring  systems  for 
assessing  each  element  of  risk  (usually  hy  task)  based  on 
historical  databases. 

•  Margin  Based  Analysis — Estimating  and  pricing  the 
potential  agreement  allowing  for  a  variety  of  risk  scenarios; 
then  adding  a  standard  margin  to  the  most  likely  or  worst 
case. 

•  Prospect  Assessiuent — ^Application  of  a  scoring  system  to  the 
user  contributions  to  risk;  then  factoring  the  pricing  by  a 
multiplier  based  on  the  assessment  of  the  prospect. 

Risk  scoring  was  cited  by  55%  of  the  respondents  as  the  preferred 
method  with  margin-based  analysis  and  prospect  assessment  being 
used  by  27%  and  18%o  of  the  respondents  respectively. 
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In  addition  to  specialized  risk  assessment  processes,  vendors  apply 
additional  tools  and  techniques  to  minimize  risk  throughout  the 
engagement.  They  tend  to  focus  on  estimating,  project 
management,  subcontractor  management  and  project  pricing. 

The  types  of  techniques  and  tools  used  for  each  of  these  areas  are 
shown  in  Exhibit  II-4. 


Techniques  Used  by  Vendors  to  Minimize  Risk  in 
Key  SI  Management  Processes 


Process 


Tool  or  Technique 


Level  of  Use 


Estimating 


Function  Point  Analysis 
Proprietary  Methodologies 
Multiple  Estimates 
Monte  Carlo  Techniques 


Project  Management 


Continuous  Tracking  and  Review 
Proprietary  Project  Mgt.  Process 
Quality  Assurance  System 


Subcontracting 


Prequalified/Preferred  Suppliers 
Full  Integration  into  Team 
Payment  or  Contract  Mechanisms 


Pricing 


Task  Estimating  Including  Risk 
Incentive  Mechanisms 
Range-based/Phase-fixed 


=  Heavy  Use 
^  =  Medium  Use 
=  Light  Use 
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As  is  the  case  with  users,  vendors  also  look  to  contract  mechanisms 
to  help  manage  risk.  In  general  these  mechanisms  are  designed  to 
increase  the  level  of  risk  sharing  on  the  part  of  the  buyer.  They 
include. 

•  Providing  for  detailed  specification  of  user  participation, 
responsibilities  and  resource  commitments  as  part  of  the 
agreement 

•  The  contractual  specification  of  detailed  management 
processes  for  change  control,  risk  management  and  quality 
assurance 

Vendors  are  also  pushing  hard  for  pricing  arrangements  which 
either  increase  risk  sharing,  or  provide  potentially  higher  rewards 
for  the  assumption  of  risk. 

•  Range-based  pricing  is  becoming  more  common.  It  allows  for  a 
range  of  prices  for  a  phase  or  task  based  on  predefined 
contingencies  which  might  arise." 

•  Phase-fixed  pricing  is  also  becoming  more  popular.  Under  this 
method  a  price  is  fixed  for  the  first  phase  and  estimates  given 
for  subsequent  phases  or  tasks.  At  the  completion  of  the  first 
fixed-price  phase,  prices  are  fixed  for  one  or  more  of  the 
subsequent  phases. 

•  When  the  financial  benefits  of  successful  implementation  are 
well  defined,  many  vendors  will  encourage  benefit  sharing 
approaches  such  as  value-based  pricing.  In  these  situations, 
the  price  may  be  fixed,  but  at  a  lower  level,  additional 
revenues  will  then  be  earned  based  on  some  portion  of  the 
actual  financial  benefits  realized  by  the  client. 

•  In  situations  where  an  effort  is  truly  leading  edge,  some 
vendors  encourage  a  joint  venture  approach,  retaining  some 
type  of  ownership  rights  to  the  technology  as  an  incentive  in 
addition  to  an  agreed  upon  price. 

Finally,  many  of  these  approaches  can  and  are  being  combined  in  a 
single  agreement,  offering  significant  opportunities  to  tailor  pricing 
and  process  with  the  risk.  In  fact,  based  on  the  survey  results, 
vendors  anticipate  a  significant  shift  from  straightforward  fixed 
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price  and  time  and  materials  agreements  to  those  incorporating 
some  of  the  mechanisms  discussed  here. 


D  

Trends  in  Risk  Management 

In  summary,  a  number  of  trends  are  emerging  with  regard  to  the 
management  of  risk  in  systems  integration  projects.  Essentially, 
these  fall  into  two  categories;  those  impacting  the  nature  of  the 
contract  relationship  between  buyers  and  vendors,  and  those 
effecting  change  in  the  SI  project  management  processes. 

The  major  trends  impacting  contracting  are: 

•  An  accelerated  shift  from  time  and  materials  pricing  to  value- 
based  or  other  incentive-based  approaches 

•  A  movement  toward  pricing  schemes  such  as  range-based  and 
phase-fixed  which  encourage  risk  sharing  (with  or  without 
incentive  clauses)  and  acknowledge  at  the  start  of  a  project 
that  there  may  be  elements  of  risk  that  simply  can  not  be 
properly  estimated  in  financial  terms. 

•  A  growing  tendency  to  include  detailed  contract  specifications 
for  user  resource  requirements  down  to  the  level  of  phase  and 
task. 

•  The  use  of  joint  venture  development  efforts  between  buyers 
and  vendors  to  deal  with  very  high-risk  projects  involving 
advanced  or  unproved  technology. 

Those  trends  impacting  SI  management  processes  include: 

•  An  increase  in  the  use  of  computer-assisted  continuous 
monitoring  processes  supported  by  on-line  computer 
applications. 

•  A  growth  in  the  use  of  specialized  quality  assurance 
assessment  teams  to  provide  early  problem  identification  and 
recommended  solutions. 
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•  An  increase  in  the  inclusion  of  user  personnel  in  quality 
assurance,  ongoing  risk  assessment  and  other  project 
monitoring  and  control  processes,  formerly  considered  internal 
to  the  vendor. 

•  A  growing  use  of  prototyping  and  application  modeling  to 
insure  user  satisfaction  with  the  end  product. 

Assuming  INPUT'S  assessment  of  these  trends  is  accurate,  they  are 
likely  to  result  in  some  fundamental  changes  in  how  risk  is 
perceived,  managed  and  priced  in  systems  integration  efforts.  ' 


E  

Conclusions  and  Recommendations 

•  While  users  still  see  fixed-price  agreements  as  the  best 
approach  to  minimizing  risk  in  SI  projects,  vendors  will 
continue  to  push  for  more  risk  sharing,  resulting  in  a  gradual 
shift  from  a  buyer/contractor  type  of  relationship  to  one  that 
functions  more  like  a  partnership.  Vendors  who  wish  to 
benefit  from  this  shift  will  need  to  be  open  to  more  innovative 
contracting  approaches. 

•  As  this  shift  occurs,  users  will  become  increasingly  integrated 
into  project  management  processes  that  once  were  considered 
internal  to  the  vendor.  Process  will  need  to  be  modified  to 
accommodate  that  integration. 

•  Over  time,  contracts  will  focus  more  on  specifying  the 
processes  to  be  used  to  manage  quality  and  risk  than  on 
specific  actions  to  be  taken  with  regard  to  a  given  project. 

•  Review,  control  and  problem  resolution  process  will  become 
virtually  real  time,  usnig  new  technology  and  systems  to 
enable  the  communication  of  issues  and  the  development  of 
adjustments  to  plan  on  an  ongoing  basis.  Some  vendors  are 
already  using  these  types  of  processes.  Others  will  need  to 
assess  and  invest  in  these  types  of  systems  to  remain 
competitive. 

•  Pricing  mechanisms  will  continue  to  adjust  to  accommodate 
the  growing  trend  toward  risk  sharing. 
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•  Over  time,  vendors  will  encourage  long-term  management 
agreements  with  their  best  clients;  and  users  will  become  more 
receptive  to  the  concept. 

-  The  majority  of  users  already  prefer  to  work  with  one  vendor, 
and  are  likely  to  give  an  existing  vendor  preferred  supplier 
status. 

-  Vendors  can  reduce  investments  in  marketing  and  startup 
expenses 

-  Both  parties  can  benefit  from  the  investment  that  they  have 
already  made  in  joint  processes 

-  Risks  are  significantly  reduced  for  future  efforts  once  the 
first  project  has  been  completed. 
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User's  Perspectives 


R 


In  general,  users  look  to  SI  vendors  as  partners  in  a  major  systems 
integration  effort.  The  implication,  of  course,  is  that  the  risks 
associated  with  the  effort  will  be  shared.  However,  for  many  user 
companies,  the  survey  data  indicates  that  little  is  done  up  front  to 
identify  and  assess  the  risks  associated  with  a  proposed  effort.  So 
how  do  users  factor  risk  into  the  partnership  equation?  This 
chapter  addresses  that  question  by  examining: 

•  How  users  perceive  the  criticality  of  various  elements  of  risk 

•  The  processes  and  contractual  mechanisms  they  put  into  place 
to  manage  the  risk 

•  Their  perspectives  on  who  should  assume  responsibility  for  the 
risk  associated  with  various  aspects  of  a  systems  integration 
engagement 

•  Their  views  on  the  key  contributors  to  quality  in  the 
partnership  arrangement  for  risk  sharing 

•  Their  preferences  for  various  approaches  to  contract  pricing 


Overall,  the  survey  identified  the  following  nine  key  areas  of  risk 
associated  with  major  systems  integration  efforts. 

•  The  initial  identification  of  requirements  by  the  user 

•  Lack  of  user  involvement  during  the  course  of  the  project 


A 


Areas  of  Risk 
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•  Project  management  by  the  SI  vendor 

•  Project  management  by  user  personnel 

•  Resource  estimating  by  the  vendor 

•  Resource  estimating  by  the  user 

•  Risk  evaluation  by  the  vendor  at  the  start  of  the  project 

•  Risk  evaluation  by  users  at  the  start  of  the  project 

•  Control  over  user  requests  for  change 

Exhibit  III-l  shows  how  significant  survey  respondents  felt  each  of 
these  sources  is  in  terms  of  impact  on  the  typical  SI  effort.  The 
values  indicate  the  average  assessment  of  the  sample  on  a  scale  of 
1-5  with  1  being  insignificant  and  5  indicating  very  significant. 


EXHIBIT  III-1 


User  Ratings  of  Key  Risk  Factors 


Risk  Factor 


User  Involvement 


User  Requirements  ^ 


Change  Control 
Risk  Eval.  -  User 
Estimating  -  User 
Project  Mgt.  -  Vendor 
Estimating  -  Vendor 
Risk  Eval.  -  Vendor 
Project  Mgt.  -  User 


1.0 
Low 


4.1 


3.8 


3.8 


3.7 


3.4 
3.3 
3.3 
3.2 


3.1 


2.0  3.0  4.0 

User  Rating  of  Each  Factor 


5.0 
High 
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The  fact  that  every  area  had  an  average  rating  greater  than  3.1 
indicates  that  users  recognize  all  nine  sources  to  be  legitimate 
areas  of  risk.  However,  the  lack  of  discrimination  between  the 
highest  rated  and  the  lowest  (4.1  to  3.1)  would  seem  to  say  that  for 
all  practical  purposes  users  see  these  sources  of  risk  as  equal  in 
terms  of  their  potential  impact. 

Grouping  the  risk  elements  in  terms  of  which  side  of  the 
partnership  has  the  majority  of  the  control  provides  some 
additional  insight.  For  example,  the  four  highest-rated 
contributors  to  risk  have  an  average  rating  of  3.9,  and  are  definitely 
factors  where  the  user  has  the  bulk  of  the  control.  For  factors  where 
the  vendor  is  dominant  the  average  is  just  under  3.3.  While  this 
difference  is  not  statistically  significant  in  a  sample  of  65,  it  does 
tend  to  indicate  that  users  recognize  that  they  have  a  significant 
impact  on  the  risk  associated  with  a  project — ^probably  more  impact 
than  the  vendor. 

This  concept  is  further  reinforced  when  the  data  is  analyzed  on  the 
basis  of  whether  the  respondent  is  a  "first  time"  user  of  SI  services 
or  has  used  systems  integrators  for  multiple  projects.  Users  with 
multiple  experiences  tend  to  rate  the  impact  of  their  contribution  to 
the  risk  at  3.8,  but  their  assessment  of  the  vendors  contribution 
drops  to  about  2.0.  Apparently,  companies  who  have  experienced 
more  than  one  major  SI  project  see  that  the  same  factors  exist  and 
acknowledge  vendors  are  perhaps  more  sophisticated  at  managing 
those  components  that  they  control. 

1.  Risk  Management — User  Perspective 

Users  employ  a  variety  of  processes  and  contract  mechanisms  to 
manage  the  elements  of  risk  associated  with  SI  efforts.  On  the 
process  side  four  key  elements  come  into  play. 

•  xA^utomated  planning  of  resource  requirements 

•  Computer-based  project  management 

•  Formalized  change  control  procedures 

•  Dedicated  user  representatives  on  the  project  team 

Exhibit  III-2  shows  the  proportion  of  respondents  who  utilized  each 
process. 
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EXHIBIT  III-2 

Use  of  Risk  Minimizing  Processes  by  Users  of  SI  Services 


Process 

Formalized 
Change  Control 


Dedicated  User 
Representatives 


Computer-based 
Project  Mgt. 


Automated 
Resource  Ping. 


0  20  40  60  80  100 

Proportion  of  Respondents  Using  Each  Response  (%) 


The  fact  that  80%  or  more  of  the  respondents  also  used 
computerized  project  management  processes  and  dedicated  staff 
indicates  that  these  approaches  have  become  more  or  less  standard 
practice.  By  comparison,  automated  resource  planning  appears  to 
be  less  popular  as  a  risk  management  tool.  This  is  probably  due  to 
the  fact  that  there  are  few  tools  on  the  market  that  adequately 
address  the  need. 

It  should  be  pointed  out  that  to  some  degree,  the  use  of  these  risk 
containment  processes  has  been  pushed  on  the  buyers  of  SI  services 
by  vendors  who  see  the  lack  of  formal  change  control  and  dedicated 
user  involvement  as  "red  flags"  in  terms  of  escalating  risk. 
Consequently,  these  processes  are  frequently  required  or  mandated 
as  part  of  the  contract. 

Users  also  employ  a  number  of  contract  mechanisms  to  mitigate 
risk.  The  most  frequently  used  include: 

•  The  inclusions  of  milestones  in  formal  contracts 
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•  Performance  clauses 

•  Guarantees  and/or  warranties  of  one  type  or  another 

•  Bonus  payments  for  early  completion  or  under-cost 
performance 

Exhibit  III-3  shows  the  proportion  of  survey  respondents  currently 
using  these  mechanisms,  as  well  as  the  proportion  who  plan  to  be 
using  them  within  the  next  two  years. 


EXHIBIT  III-3 

Contract  Mechanisms  Used  to  Minimize  Risk  by  Users-Current  and  Future 


Mechanism 


Milestones 


Performance 
Clauses 


Warranties 


Bonuses 


20 


94 


91 


88 


83 


85 


76 


32  S  Future  Use 

□  Current  Use 


0  20  40  60  80 

Proportion  of  User  Respondents  (%] 

No.  of  Respondents  =  65 


100 


With  regard  to  contract  mechanisms,  the  inclusions  of  specific 
milestones  and  performance  clauses  (mostly  related  to  task 
completions)  appear  to  be  a  standard  practice. 

Although  the  use  of  bonus  mechanisms  was  the  least-frequently 
cited  contract  risk  containment  mechanism,  the  survey  data 
indicates  that  it  will  experience  a  60%  increase  in  usage  over  the 
next  two  years.  A  number  of  factors  are  encouraging  this  growth 
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•  Vendors  continue  to  push  for  more  incentive-based  pricing. 
Bonus  payments  for  early  completion,  etc.,  provide  a 
straightforward  mechanism  for  accomplishing  that  objective. 

•  A  growing  number  of  systems  integration  projects  are  the 
direct  result  of  business  process  reengineering  initiatives. 
When  this  is  the  case,  many  vendors  tend  to  push  for  value- 
based  pricing  or  bonus  payments  by  tying  their  revenues  for  SI 
services  directly  to  the  financial  benefits  achieved  as  a  result 
of  the  business  reengineering  effort. 

•  Most  users  perceive  the  relationship  with  a  systems  integrator 
as  a  partnership  where  risks  will  be  shared.  Thus,  sharing  the 
financial  rewards  in  the  form  of  bonus  or  incentive  payments 
is  a  logical  way  to  reward  exceptional  performance  on  the  part 
of  the  integrator. 

While  we  can  anticipate  that  use  of  financial  incentives  will  become 
a  growing  part  of  SI  agreements,  it  should  be  pointed  out  however, 
that  contracts  that  use  these  mechanisms  almost  invariably 
balance  the  "carrot"  with  the  "stick".  Of  the  20  respondents  who 
indicated  that  bonus  payments  had  been  part  of  their  contracts,  19 
indicated  that  these  were  balanced  by  penalty  payments  of  some 
type  or  another. 

Finally,  several  respondents  commented  that  even  though  they 
currently  used  financial  incentives  as  a  key  contract  mechanism  for 
risk  management,  the  process  had  to  be  managed  carefully.  Unless 
the  definition  of  the  reward  and  the  processes  for  establishing 
eligibility  are  carefully  spelled  out,  debate  over  qualification  for  the 
award  or  its  amount  could  quickly  reduce  a  partnership  to  an 
adversarial  relationship. 


B  

Working  with  ST  Vendors 

The  study's  findings  support  the  concept  that  while  an  SI 
engagement  is  a  partnership,  users  expect  vendors  to  take  prime 
responsibility  for  the  overall  effort.  Exhibit  III-4  shows  how  users 
rate  the  importance  of  their  roles,  and  those  of  vendors  in  terms  of 
assuming  responsibility. 
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EXHIBIT  III-4 

User  Importance  Ratings  for  Levels  of  Project  Responsibility 


Level  of 
Responsibility 


Prime 


Selected 
Elements 


Support 
Services 


1.0         2.0         3.0         4.0  5.0 

Low  High 
User  Rating  of  Importance 


/ 

The  results  can  be  interpreted  as  follows: 

•  The  4.3  for  vendors,  in  terms  of  assuming  prime  responsibility 
for  the  completion  of  the  contract,  is  consistent  with  the  notion 
that  the  main  reason  for  hiring  an  integrator  is  to  provide 
overall  leadership  in  the  effort  from  the  planning  stages 
through  implementation. 

•  The  relatively  high  3.9  importance  rating  for  "selected 

,  elements"  for  vendors  and  users  reflects  the  partnership  aspect 
of  the  arrangement.  In  other  words,  within  the  overall 
framework  of  the  vendor-managed  project,  responsibility  for 
various  subtasks  may  be  assigned  to  either  partner,  and  it  is 
relatively  important  that  the  assignments  be  clear. 
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•  The  lower  ratings  for  support  services  indicate  that  from  a 
users  perspective  either  partner  can  assume  responsibiUty  for 
these  services.  However,  it  is  not  nearly  as  important  as 
establishing  overall  responsibility  with  the  vendor  and 
insuring  that  clear  responsibility  for  subtasks  is  assigned  to 
one  party  or  the  other. 

Exhibit  III-5  indicates  how  users  rate  the  importance  of  vendors 
assuming  responsibility  for  various  components  of  a  typical  systems 
integration  project. 


EXHIBIT  III-5 

User  Ratings  of  Desireability  of  Vendor  Assumption  of  Responsibility 


Responsibility 


Development 
Detail  Design 
Conformance 
System  Tuning 
implementation 
.  User  Training 
Cost 

Performance  Meas. 

Functional  Specs 
BPR 


1.0 
Low 


3.7 


2.9 


2.0  3.0  4  0 

User  Rating  of  Desireability 


5.0 
High 


Development,  detail  design,  implementation,  user  training,  cost 
and  performance  measurement  should  be  pretty  much  self- 
explanatory.  Some  of  the  other  responsibility  areas  shown  in  the 
exhibit  warrant  additional  explanation. 


1-8 


i3  1994  by  INPUT  Reproduction  Prohibited. 


BIRR 


CONTRACTUAL  APPROACHES  TO  PROJECT  RISK  REDUCTION 


INPUT 


•  Conformance  as  used  here  means  that  the  final  product  or 
system  meets  the  planned  business  needs. 

•  System  tuning  is  the  process  of  making  modifications  to  the 
finished  systems  to  improve  operational  performance. 

•  Functional  specifications  in  the  context  of  this  study  should  be 
interpreted  as  the  activity  of  creating  the  specifications. 

•  BPR  is  the  abbreviation  for  business  process  reengineering. 

Given  these  definitions,  there  are  a  number  of  observations  that 
can  be  made  about  the  relative  importance  users  attach  to  the  role 
of  vendors  in  terms  of  responsibility  for  various  aspects  of  the 
project. 

With  the  exception  of  BPR,  users  seem  to  think  that  vendors  should 
assume  some  significant  level  of  responsibility  in  most  of  the 
activities  associated  with  the  project.  This  view  is  certainly 
consistent  with  the  earlier  observation  that  they  want  vendors  to  be 
responsible  for  the  overall  effort.  However,  there  does  not  appear  to 
be  a  lot  of  significant  variation  between  specific  types  of 
responsibility.  Average  ratings  varied  only  1  point  between  2.9  and 
3.9  across  the  various  categories. 

There  does  appear  to  be  one  grouping  of  activities  where  users 
clearly  see  the  vendor  as  the  primary  responsible  party.  These 
activities  all  relate  to  development  and  implementation.  Design, 
development,  tuning  and  implementation  all  received  average 
ratings  in  excess  of  3.7.  Interestingly  enough,  in  the  area  of  BPR 
and  the  creation  of  functional  specifications,  users  do  not  rate  the 
desirability  of  vendor  participation  nearly  as  high  (2.9  and  3.0 
respectively).  Yet,  these  are  the  areas  where  vendors  are  promoting 
their  expertise.  This  is  probably  attributable  to  the  fact  that 
despite  the  amount  of  press  on  BPR,  most  SI  agreements  are  still 
initiated  at  the  point  where  some  broad  functional  specification  has 
already  been  developed,  either  in-house,  or  as  the  result  of  some 
prior  BPR  effort.  In  effect,  it  is  too  early  in  the  evolution  of  BPR  as 
part  of  the  systems  process  to  get  an  accurate  measurement. 
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Evaluation  of  the  Relationship 


The  research  indicates  that  most  companies  perceive  the  quahty  of 
the  partnership  relationship  between  their  organization  and  their 
systems  integrator  to  be  above  average.  The  average  rating  for  the 
sample  indicated  a  satisfaction  level  of  3.7  on  a  scale  of  1-5,  with  5 
high.  However,  the  same  group  felt  that  there  were  lots  of 
opportunities  for  improvement,  rating  the  desirability  for 
improvement  in  the  relationship  at  4.4  on  the  same  1-5  scale.  , 

Users  represented  in  the  survey  identified  three  key  areas  that  are 
critical  to  success  in  the  partnership  including:  a  vendor's 
understanding  of  their  business,  the  willingness  of  the  vendor  to 
assume  risk  and  cultural  compatibility. 

As  shown  in  Exhibit  III-G,  business  understanding  is  key  from  the 
users  perspective. 


EXHIBIT  III-6 


User  Ratings  of  Factors  Contributing  to  Partnership  Success 


Factor 


Vendor  Bus. 
Knowledge 


Vendor  Risk 
Assumption 


Common 
Culture 


3.0 


%  Requiring 
Improvement 

71 


1.0  2.0  3.0  4.0  5.0 

Low  High 
User  Rating  of  Contributing  Factors 


63 


52 


The  fact  that  cultural  fit  rated  significantly  lower  than  the  other 
factors  is  partially  explainable  by  the  fact  that  74%  of  the  sample 
had  already  adopted  one  or  more  preferred  suppliers  of  SI  services. 
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Thus,  cultural  fit  and  familiarity  with  the  partner  was  already 
established. 

Exhibit  III-6  also  shows  the  percentage  of  respondents  who  felt  that 
improvement  could  be  made  in  each  area.  Interestingly,  the  need 
for  improvement  correlates  directly  with  the  ratings  on  each  factor. 
This  reinforces  the  notion  that  in  most  instances  there  is  room  for 
improvement  in  the  key  areas  critical  to  the  success  of  most 
vendor/user  partnerships. 

As  shown  in  Exhibit  III- 7,  most  users  feel  that  the  addition  or 
modification  of  some  contract  components  would  help  improve  the 
partnership. 


EXHIBIT  III-7 


Proportion  of  Users  Who  Feel  Relationships 
Could  be  Improved  by  Contract  Changes 


Milestones 


Performance  Clauses 


Guaranties  or  Warranties 


Bonus  Payments 


56 


89 


92 


90 


0        10       20       30       40       50       60       70       80       90  100 

Proportion  of  Respondents  (%) 


As  discussed  earlier,  most  respondents  already  use  milestones, 
performance  clauses  and  guaranties  or  warranties  to  reduce  or 
mitigate  risk.  The  fact  that  approximately  90%  of  the  sample  rates 
these  three  mechanisms  as  key  to  improving  the  partnership 
implies  that  their  presence  in  the  agreement  helps  clarify  the 
understanding  between  the  vendor  and  user  at  the  beginning  of  the 
effort,  without  setting  up  an  antagonistic  relationship. 
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While  only  20%  of  the  respondents  indicated  that  they  currently 
use  bonus  payments  or  other  financial  incentives  in  their  current 
agreements,  56%  feel  that  the  addition  of  these  types  of  contract 
mechanisms  would  help  to  improve  the  quality  of  the  partnership. 
This  reinforces  the  idea  that  users  are  becoming  more  receptive  to 
this  type  of  contractual  agreement  and  will  most  likely  promote  the 
concept  more  heavily  over  time. 

Finally,  despite  the  issues  associated  with  dealing  with  risk,  the 
vast  majority,  as  shown  in  Exhibit  III-8,  seem  to  feel  that  the  risk 
is  being  addressed  adequately. 


EXHIBIT  III-8 

User  Levels  of  Satisfaction  with  Risk  Containment 


Low  High 


No.  of  Respondents  =  65 


D  

The  Impact  on  Pricing 

1.  General  Views  on  Pricing  Strategy 

Users  perceptions  about  risk  are  reflected  in  their  preferences  to 
various  approaches  to  contract  pricing.  As  shown  in  Exhibit  III-9, 
users  clearly  prefer  fixed-price  contracts  when  possible. 
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EXHIBIT  III-9 

User  Preferences  for  Pricing  Mechanisms 

Mechanism 


Fixed  Price 


Time  and 
Materials 


Value-based 


1.0  2.0  3.0  4.0  5.0 

Low  High 
User  Preference  Rating 


Fixing  the  price  at  the  beginning  of  the  project,  at  least  on  the 
surface,  certainly  reduces  the  financial  risk  associated  with  an  SI 
effort,  and  clearly  appears  to  he  a  reasonable  strategy  when  the 
scope  of  the  effort  is  extremely  well  defined.  However,  it  does  have 
some  disadvantages. 

•  In  situations  where  the  vendor  senses  high  levels  of  risk  that 
are  either  denied  or  ignored  by  the  prospect,  vendors  will 
factor  the  risk  into  the  pricing,  thus  increasing  the  cost  to  the 
buyer. 

•  Fixing  the  price  will  clearly  reduce  the  vendor's  interest  in 
being  flexible  with  regard  to  changes  in  specifications,  etc. 

•  When  unanticipated  events  develop  during  the  project  that 
require  midcourse  adjustments  in  the  level  of  resources  or 
changes  in  schedules,  the  fixed-price  approach  can  be  a  barrier 
to  effective  negotiations — turning  the  partnership  into  a 
potentially  adversarial  relationship. 
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Advocates  of  the  time-and-materials  approach  (average  user  rating 
of  3.1)  say  that  the  obvious  risk  of  significant  overruns  in  cost  can 
be  managed  using  other  types  of  contract  mechanisms. 

•  They  cite  milestones  tied  to  task  schedules  as  one  technique  of 
controlling  cost,  and  use  processes  jointly  agreed  to  prior  to  the 
initiation  of  the  project  to  assess  early  warning  conditions  for 
potential  overruns  or  other  deviations  from  plan. 

•  Another  approach  recommended  by  some  users  is  fixing  the 
price  by  major  project  phase.  For  example,  an  SI  project  that 
covers  the  generation  of  functional  specifications  through 
implementation,  and  user  training  may  be  planned  as  a  four- 
phased  effort.  The  contractual  arrangement  will  provide  for  a 
fixed  price  for  the  first  phase,  and  estimated  prices  for  phases 
two  through  four.  At  the  onset  of  phase  two,  the  price  will  be 
fixed,  and  so  on  throughout  the  remainder  of  the  project.  In 
these  types  of  agreements  the  contract  may,  or  may  not, 
provide  for  a  fixed-ceiling  price  for  the  subsequent  phases. 

A  number  of  users  also  indicated  that  they  preferred  a  mixture  of 
the  fixed-price  and  time-and-materials  approaches.  In  situations 
where  the  vendor  and  the  buyer  see  a  particular  phase  or  subtask 
of  the  effort  as  having  significant  unknowns,  they  may  agree  to 
time-and-materials  for  that  phase  or  task,  and  use  fixed-pricing  for 
other  parts  of  the  effort. 

Clearly  from  a  buyers  ])erspective,  fixed-pricing  and  time-and- 
materials,  or  some  combination  of  the  two,  are  the  most  popular 
pricing  mechanisms  today.  The  average  of  user  ratings  for  value- 
based  pricing  was  only  1.7  on  the  1-5  scale.  However,  as  already 
pointed  out,  other  data  gathered  in  the  survey  indicates  that  value- 
based  approaches  are  likely  to  increase  in  popularity  over  time. 

2.  Perceived  Advantages  of  Each  Pricing  Approach 

Despite  the  preference  for  fixed-price,  users  see  some  advantages  to 
each  of  the  three  major  categories  of  pricing.  The  following  sections 
list  the  advantages  users  cited  for  each  method.  The  advantages 
are  listed  in  descending  order  based  on  their  frequency  of  mention 
by  survey  respondents. 
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a.  Fixed-Price 

•  Budget  Management — Clearly  the  dominant  advantage  cited 
by  users  for  fixed-price  approaches  was  the  fact  that  it 
provides  them  with  a  way  to  guarantee  the  budget  for  the 
effort.  Besides  the  obvious  advantage  of  providing  a  fix  on  the 
budget,  users  indicated  that  fixed-pricing,  allowed  for  more 
effective  comparison  of  competitive  bids,  and  made  it  easier  to 
sell  projects  to  executive  management. 

•  Risk  Management — ^This  was  the  second  most  frequently  cited 
advantage  to  fixed-pricing.  Respondents  expressed  this 
advantage  in  a  number  of  ways.  Some  indicated  that  "for  a 
price"  it  met  their  objective  of  forcing  the  majority  of  the  risk 
on  the  vendor.  Other's  indicated  that  it  forced  the  user  and 
vendor  to  a  more  careful  examination  of  project  objectives, 
scope  and  specifications  in  advance — reducing  the  overall  risk. 

•  Project  Control — ^Although  third  in  the  ranking,  at  least  10 
respondents  believed  that  using  fixed-priced  contracts 
encouraged  vendors  to  keep  projects  on  track  by  placing  the 
burden  of  resource  and  schedule  management  directly  on  the 
vendor. 

b.  Time-and-Materials 

•  Cost — ^This  was  the  most  frequently  cited  advantage. 
Essentially,  users  feel  that  a  properly  managed  time-and- 
materials  contract  will  yield  them  the  lowest  cost  for  a  large 
variety  of  projects,  by  eliminating  the  "padding"  that  vendors 
include  in  fixed-price  deals  to  cover  the  risk.  To  quote  one 
respondent,  "You  only  pay  for  what  you're  getting." 

•  Flexibility — significant  number  of  users  felt  that  the  ability 
to  change  course  or  refine  specifications  during  the  project 
could  only  be  accomplished  through  the  use  of  time-and- 
materials  contracts.  They  expressed  concern  over  the  quality 
of  the  results  in  situations  where  fixed-pricing  was  used  and 
changes  needed  to  made  midstream. 
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c.  Value-based 

Only  12  respondents  elected  to  comment  on  the  advantages  of 
value-based  pricing.  Their  comments  indicate  that  the  primary 
advantages  of  this  approach  are: 

•  It  creates  and  environment  to  support  more  visionary  thinking 
about  overall  benefits  to  the  buyer. 

•  It  encourages  "partnership"  thinking.  Working  together  to 
obtain  a  financial  object  that  will  directly  benefit  both  parties 
encourages  rapid  and  creative  problem  resolution. 

3.  Preferred  Pricing  by  Type  of  Activity 

Survey  respondents  were  asked  to  provide  information  on  their 
preferences  for  pricing  strategy  as  a  function  of  task  type.  Exhibit 
III-IO  shows  the  frequency  of  mention  for  various  types  of  tasks  for 
fixed-price  contracts. 

Design  and  development  activities  dominate  the  group, 
representing  just  over  50%  of  the  responses.  Eight  respondents 
commented  that  they  used  fixed-price  contracting  for  virtually  all  of 
information  systems  work  contracted  to  vendors.  Five  of  those  who 
indicated  that  they  used  fixed-price  contracting  for  design  and 
development  activities  added  the  caveat  that  they  only  did  so  for 
well-defined  projects. 

Approximately  30%  indicated  that  they  contracted  consulting  on  a 
fixed-priced  basis  as  well,  but  the  majority  undertook  consulting 
projects  on  a  time-and-materials  basis. 

But  essentially  the  data  does  not  delineate  any  specific  preferences 
for  contract  price.  Other  than  indicating  a  strong  general  tendency 
to  prefer  fixed-priced  contracting  for  design  and  development 
efforts,  most  users  indicated  that  they  would  use  any  of  the  three 
methods  depending  on  the  characteristics  surrounding  the 
particular  systems  integration  project. 
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EXHIBIT  111-10 


User  Preferences  for  Fixed-Price  Contracts  by  Type 


Task 


Design 
Development 


Consulting 


Standalone 
Implementation 

Hardware 
Implementations 


10        15  20 
Number  of  Responses 


25 


30 


Furthermore,  users  make  little  differentiation  on  the  type  of 
contracting  used  based  on  the  size  of  the  project.  Rankings  by 
survey  respondents  were  identical  for  large  and  small  projects  with 
fixed-price  ranking  first,  and  time-and-materials  and  value-based 
pricing  ranking  second  and  third  respectively. 

In  summary,  it  seems  clear  that  risks  do  have  an  impact  on  users 
preferences  for  pricing  strategies,  at  least  for  the  moment,  along 
with  increasing  the  emphasis  on  fixed-price  deals.  Nevertheless, 
users  are  open  to  creative-pricing  arrangements,  and  will  adopt  a 
different  pricing  approach  depending  upon  the  situation.  This 
willingness  to  adapt  their  strategy  is  strongly  motivated  by  a  belief 
that  whatever  pricing  mechanism  is  used,  it  should  encourage  a 
stronger  partnership  relationship. 

This  indicates  that  the  industry  should  continue  to  move  to 
creative-pricing  concepts  that  are  tailored  to  the  particular 
vendor/client  situation. 
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E  ^  

Preferred  Suppliers 

Seventy-four  percent  (74%)  of  the  survey  respondents  had  used  the 
same  systems  integrator  on  two  or  more  projects,  and  felt  that 
doing  so  had  a  significant  impact  on  reducing  risk.  They  cited  long- 
term  relationships,  established  communications  modes,  personal 
relationships,  mutual  goals  and  common  experience  as  key  factors 
in  the  risk  reduction  equation.  Exhibit  III- 11  shows  the  frequency 
of  mention  of  the  reasons  that  users  feel  multiple  engagements 
reduce  risk. 


EXHIBIT  111-11 

User  Reasons  for  Using  Preferred  Suppliers 


Reason 

Existing 
Knowledge 

Process 
Compatibility 

Cultural 
Compatibility 

Reduced  Start 
Up  Time 

Mutual  Trust 

r 

1                             1                             1                             1  1 

0           5           10          15          20  25 
Number  of  Mentions 

Clearly  the  common  linkage  for  all  of  these  factors  is  the  "learning 
curve"  phenomena.  There  is  a  lot  to  be  said  for  familiarity  as  a 
sound  basis  for  building  an  on-going  relationship. 

Even  one  out  of  three  of  the  26%  of  the  respondents  who  indicated 
that  they  did  not  use  "preferred  suppliers"  said  there  were  clearly 
advantages  to  using  a  supplier  they  had  previously  worked  with, 
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but  felt  that  designating  a  supplier  as  "preferred"  would  reduce 
their  ability  to: 

•  Get  the  best  price  for  each  deal  as  it  arose 

•  Gain  the  benefit  of  new  approaches  or  technologies  that  a  new 
supplier  might  be  able  to  bring  to  a  particular  situation 

The  evidence  from  this  part  of  the  study  certainly  sends  a  signal  to 
vendors.  Top-notch  performance  on  the  first  job  with  a  client  will 
go  a  long  way  toward  insuring  future  engagements,  reducing 
marketing  expense  and  having  a  positive  impact  on  risk  reduction 
over  time. 


F  

Overall  Attitudes 

Analysis  of  data  gathered  during  the  survey  on  overall  user 
attitudes  provides  a  good  vehicle  for  summarizing  this  chapter  on 
user  perspectives.  Survey  respondents  were  asked  to  rate  the 
extent  to  which  they  agreed  or  disagreed  with  ten  statements  about 
systems  integration  risk  management  and  contract  issues.  A  rating 
of  1  indicated  strong  disagreement,  and  a  5,  strong  agreement. 
Exhibit  111-12  shows  the  statements  and  average  ratings  for  the 
survey  sample. 


BIRR 


©  1994  by  INPUT  Reproduction  Prohibited 


111-19 


CONTRACTUAL  APPROACHES  TO  PROJECT  RISK  REDUCTION  INPUT 


EXHIBIT  111-12 

User  Agreement  Levels  with  Selected  Statements 
Regarding  SI  Risk  Management  and  Contract  Issues 


statement 

Agreement 
Rating* 

Improved  partnerships  would  lead  to  improved  ability  to  meet 
the  client's  business  needs 

4.4 

Vendor  profitability  is  adequate  to  cover  their  risk 

4.1 

Vendors  should  take  more  responsibility  for  project  risk 

3.9 

Client  and  vendor  should  share  the  risks  and  the  rewards 

3.8 

A  fixed-price  project  is  essential  for  budgeting  purposes 

3.8 

Clients  should  take  an  equal  share  of  project  risk 

3.7 

Vendors  should  not  be  offered  incentives  beyond  fixed-price 

3.3 

Fixed-price  projects  lead  to  an  adversarial  relationship 

3.2 

Value-based  pricing  would  lead  to  increased  project  success  ■ 

3.0 

Vendor  incentives  should  be  based  on  a  value-based 
component 

3.0 

*  1  =  Low,  5  =  High 

Based  on  these  ratings  there  appears  to  be  no  question  that  users 
beheve  that  improved  partnerships  will  go  a  long  way  toward 
meeting  their  needs  and  reducing  risks.  Yet,  beyond  that  there 
seems  to  be  little  flexibility  about  how  contract  mechanisms, 
beyond  fixed  price,  can  be  used  to  achieve  that  objective. 

In  the  minds  of  the  buyers  fixed-price  contracting,  the  preferred 
method  of  pricing  for  the  majority  of  survey  respondents,  already 
includes  a  premium  to  the  vendor  to  cover  the  risk.  Therefore,  its 
not  surprising  that: 

•  They  do  not  feel  strongly  that  additional  incentives  are 
warranted,  or  would  increase  the  probability  of  success. 
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•  They  agree  that  cHents  and  vendors  should  share  in  the  risk, 
and  are  ah-eady  doing  that  through  the  premium  payment  for 
fixed  price. 

As  more  innovative  contracting  mechanisms  are  tried,  undoubtedly 
users  may  begin  to  see  the  advantages  of  (lower-priced)  fixed  plus 
value-based  payment  schemes.  In  the  meantime,  these  more 
innovative  approaches  are  likely  to  be  acceptable  to  a  less  than  25% 
of  the  buying  community. 
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Vendors'  Perspectives  on  Risk 


Users  are  exposed  to  the  risk  associated  with  systems  integration 
projects  on  an  engagement-by-engagement  basis.  And  while  the 
impact  of  a  "project  gone  bad"  can  be  damaging  from  a  business 
perspective,  it  is  unhkely  to  pose  a  serious  long-term  threat  to  the 
user's  core  business.  Furthermore,  in  all  but  a  handful  of  cases, 
their  vendor  partners  will  persevere,  regardless  of  the  severity  of 
the  situation,  in  order  to  deliver  on  their  commitments. 

On  the  other  hand,  managing  risk  effectively  on  a  project-by-project 
basis  is  a  vendor's  lifeblood,  impacting  the  bottom  line  through: 

•  Cost  overruns  caused  by  unanticipated  problems  encountered 
in  an  engagement,  particularly  if  it  is  fixed-price 

•  The  potential  damage  to  a  vendor's  reputation  resulting  from  a 
failed  project,  and  its  negative  impact  on  future  revenue 
streams 

Consequently  vendors  tend  to  take  a  much  more  thorough  and 
sophisticated  approach  to  identifying  and  managing  the  risk  in  a 
potential  SI  engagement. 

This  chapter  examines: 

•  Vendors'  assessments  of  the  importance  of  the  various  risk 
elements  associated  with  SI  engagements 

•  Their  methods  for  assessing  overall  risk  for  individual  projects 

•  The  contracting,  pricing  and  management  processes  they  use 
to  control  exposure  to  risk 
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Measuring  Risk 


1.  Key  Risk  Factors — ^Vendor's  Perspectives 

Vendors  were  asked  to  give  their  ratings  on  the  1-5  scale  of  the 
same  nine  sources  of  risk  that  were  rated  by  user  respondents.  The 
results  showing  the  average  ratings  of  the  vendors  are  contained  in 
Exhibit  IV- 1 


EXHIBIT  IV-1 


Vendor  Ratings  of  Key  Risk  Factors 


Risk  Factor 


Complexity  of  Technology 
Change  Control 

Project  Mgt.  by  User 

User  Requirements 

User  Involvement 

Estimating  by  Vendor 

Estimating  by  User 

Project  Mgt.  by  Vendor 

Risk  Eval.  by  Vendor 

Risk  Eval.  by  User 


i 

4.2 
4.0 


4.0 

3.9 


3.8 


3.7 
3.6 


3.5 


3.5 


3.1 


1.0 
Low 


2.0  3.0  4.0 

Vendor  Rating  for  Each  Factor 


5.0 
High 


As  was  the  case  with  the  user  respondents,  vendors  saw  all  the 
sources  as  relatively  important  with  average  ratings  for  each  of  3.1 
or  greater. 
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Furthermore,  vendors  rated  the  potential  sources  of  risk  associated 
with  the  buying  organization,  or  user,  as  most  critical.  In  that 
regard,  these  ratings  are  consistent  with  those  obtained  from  the 
user  survey.  However,  there  are  two  areas  where  the  perceptions  of 
users  and  vendors  differ  significantly. 

•  Vendors  rate  user  project  managers  as  a  much  more 
significant  source  of  risk  than  do  users.  Vendor  respondents 
show  it  tied  for  first  place  compared  to  users,  who  as  a  group, 
ranked  it  last.  This  is  probably  because  users  and  vendors  see 
the  role  of  the  user  project  manager  quite  differently.  Users 
perceive  the  role  as  one  of  contract  manager.  Vendors  look  to 
the  user  project  manager  as  the  chief  source  of  liaison  to  the 
buyer  organization — ^the  key  player  on  the  user  side  once  the 
contract  has  been  signed.  They  look  to  this  individual  not  only 
to  be  well  informed  about  the  project  but  posessing  the  ability 
and  authority  to  make  day-to-day  decisions,  as  well  as  being 
connected  to  the  key  executive  management  of  the  buying 
company.  From  the  perspective  of  vendors,  a  user  project 
manager  without  those  attributes  can  rapidly  become  a  road 
block  and  consequently,  a  significant  risk. 

•  Another  area  where  the  users  and  vendors  tend  to  have  a 
different  view  is  risk  estimating.  Users  tend  to  see  their  role 
in  that  process  as  a  greater  source  of  risk  than  the  estimating 
of  the  vendor.  Vendors  take  exactly  the  opposite  view,  most 
likely  because: 

-  Despite  the  use  of  some  relatively  sophisticated  risk 
assessment  techniques,  vendors  are  betting  their  profitability 
on  the  project  when  they  use  their  risk  estimates  to  factor 
pricing, 

-  In  the  end,  the  vendor's  assessment  of  the  risk  is  likely  to  be 
controlling.  Regardless  of  the  users  assessment,  the  vendor 
is  expected  to  assume  the  lion's  share  of  the  risk. 

Finally,  the  majority  of  the  vendors  identified  and  rated  above  4.0, 
the  complexity  of  the  technology  required  for  project.  This 
indicated  that  the  risk  was  frequently  overlooked  or 
underestimated  by  the  buyer.  Perhaps  the  best  way  to  sum  up 
their  collective  view  is  by  quoting  one  respondent  who  said, 
"Executives  are  beginning  to  believe  what  they  read  in  airline 
magazines  and  the  trade  press...  and  are  coming  away  with  the 
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false  impression  that  the  technology  part  of  the  equation  contains 
few  unknowns." 

At  least  six  of  the  eleven  vendors  felt  this  user  dismissal  of  the 
technology  as  a  significant  risk  factor  was  a  growing  problem  and 
could  haunt  the  success  of  project  from  beginning  to  end  unless 
identified  and  discussed  openly  on  the  front  end.  The  hype  about 
client/server  is  a  contributing  factor.  To  quote  a  respondent,  "Never 
have  so  many  thought  they  could  do  so  much  with  a  few  PCs  and  a 
LAN."   Vendors  also  acknowledge  that  they  contribute  directly  to 
the  problem  in  the  way  they  package  and  present  their  offerings. 
At  least  four  indicated  that  the  trend  toward  selling  the  high-end 
services  such  as  BPR,  and  placing  emphasis  on  methodologies  and 
tools,  is  a  contributing  factor.  It  leaves  users  with  the  impression 
that  once  all  those  wonderful  processes  have  been  applied,  code 
development  and  implementation  is  a  snap. 

So  while  users  and  vendors  generally  agree  on  the  sources  of  risk, 
their  respective  roles  in  the  SI  partnership  cause  variations  in  their 
perceptions  about  the  significance  of  specific  elements  of  risk.  The 
way  in  which  vendors  address  these  risk  issues  is  discussed  later  in 
this  chapter. 

2.  Risk  Situation  Assessment 

There  is  clearly  a  limit  of  risk  beyond  which  a  vendor  will  refuse  to 
act  on  a  potential  opportunity.  However,  approaches  for  assessing 
the  risk,  determining  whether  or  not  to  pursue  and  opportunity 
vary  from  vendor  to  vendor. 

On  average,  the  vendors  included  in  the  survey  turned  away  20%  of 
their  potential  opportunities  on  the  basis  of  risk.  But  this  number 
is  difficult  to  assess  due  to  some  widely  varying  practices  for 
soliciting  business. 

•  Some  of  the  vendors  said  they  would  go  to  preliminary 
proposal  stage  on  literally  any  project  that  they  felt  they  had 
the  competency  to  undertake.  Consequently,  they  back  away 
from  30  to  50%  of  the  opportunities  they  pursue. 


IV-4 


©  1994  by  INPUT  Reproduction  Prohibited. 


BIRR 


CONTRACTUAL  APPROACHES  TO  PROJECT  RISK  REDUCTION 


INPUT 


't 
) 

•  Others  indicated  that  they  hardly  "put  pencil  to  paper"  on 
many  of  the  opportunities  they  see,  due  to  their  preliminary 
subjective  assessment  of  the  risks  involved.  Therefore,  they 
only  back  away  from  under  5%  of  the  opportunities  that 
actually  go  to  the  proposal  stage. 

Although  the  actual  detailed  process  of  risk  assessment  varies 
significantly,  the  vendors  surveyed  tended  to  use  one  of  three 
approaches.  Exhibit  IV-2  shows  the  proportion  of  respondents 
using  each  approach. 


EXHIBIT  IV-2 


Vendors'  Use  of  Various  Approaches  for  SI  Project  Risk  Assessment 


Prospect 
Assessment 

18%^^ 

Formal  Risk 

Margin-based 

Scoring 
55%' 

Analysis 
27% 

Number  of  Vendors  =  1 1 

Proportion  of  Vendors 

a.  Formal  Risk  Scoring 

Formal  risk  assessment  involves  the  assignment  of  point  scores  to 
various  elements  of  the  proposed  project  to  reflect  risk.  The  scores 
are  totaled  to  establish  an  overall  risk  for  the  project.  In  some 
instances  this  is  done  on  a  task-by-task  basis.  In  others,  the  project 
is  scored  in  its  entirety.  Once  the  high  risk  tasks  or  aspects  of  the 
project  have  been  identified,  most  vendors  apply  additional  analysis 
to  assess  the  risk  in  more  detail  and  develop  strategies  to  reduce. 
The  overall  scoring  is  then  adjusted  to  reflect  the  revisions.  Some 
vendors  actually  conduct  this  analysis  phase  jointly  with  the 
prospect.  By  doing  so,  they  establish  a  joint  understanding  with 
the  client  of  the  risk  and  set  mutually  agreed  to  expectations  for 
what  would  be  required  to  contain  it. 
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Once  the  scoring  process  has  been  completed,  most  vendors  apply 
guidelines  or  standards  to  assist  them  in  determining  whether  they 
are  willing  to  accept  the  risk  level  indicated  by  the  scoring. 

The  kinds  of  factors  considered  in  risk  scoring  techniques  cover  a 
wide  variety  of  issues  related  to  the  project  itself,  as  well  as  the 
prospect.  The  following  is  a  composite  sample  of  the  types  of  factors 
vendors  mentioned  as  candidates  for  risk  scoring. 

•  The  overall  length  of  the  project 

•  The  complexity  of  the  technology  required 

•  The  proposed  pricing  scheme 

•  The  prospect's  clarity  on  specifications 

•  Previous  experience  with  the  industry  or  application  suite 

•  An  assessment  of  the  strength  of  the  prospect's  project 
coordinator 

•  Level  of  competency  of  the  prospect's  IS  personnel 

•  Willingness  of  the  prospect  to  utilize  the  vendor's  methodology 

•  Cultural  fit 

•  Previous  experience  with  the  prospect 

•  Experience  of  the  vendor  project  manager 

Since  many  of  the  factors  require  subjective  evaluation,  most 
companies  who  use  risk  scoring  have  developed  formal  guidelines 
based  on  historical  data  to  make  these  judgments.  The  scoring 
guidelines  are  updated  periodically  to  reflect  history  from  recent 
projects  using  a  history  database. 

A  number  of  different  processes  are  used  to  make  the  final  go/no-go 
decision.  One  firm  indicated  that  it  did  not  bother  to  score  projects 
under  a  certain  size  and  applied  scoring  only  to  projects  it 
considered  highly  risk-prone.  Two  others  said  they  used  the  process 
on  all  projects.  Another  indicated  that  they  used  the  risk  scoring  as 
a  part  of  a  process  for  escalating  the  go/no-go  decision  within  their 
firm.  When  scores  exceed  a  certain  level,  the  project  is  submitted  to 
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a  more  senior  level  or  review  committee  to  obtain  approval  for  a  go- 
ahead. 

b.  Margin-based  Analysis 

This  approach  uses  resource  costs  and  volumes  as  the  key 
parameters  in  assessing  the  risk  associated  with  a  project.  The 
scheme  works  as  follows: 

•  An  overall  project  plan  at  the  task  level  is  prepared  for  the 
project  and  the  firms  standard  pricing  applied  by  task. 

•  An  independent  risk  assessment  team  examines  the  proposed 
pricing,  and  through  interaction  with  the  proposal  team, 
identifies  high  risk  tasks. 

•  Adjustment  factors  (multipliers)  are  agreed  to  by  the  risk 
assessment  and  proposal  teams  then  applied  to  each  task.  The 
result  is  a  "worst  case  cost"  scenario. 

•  A  standard  margin  is  applied  to  determine  the  final  price  and 
an  assessment  is  made  as  to  whether  the  resulting  price  will 
be  acceptable  (or  within  an  acceptable  bandwidth  in 
competitive  bidding  situations). 

/  •  Assuming  a  positive  judgment  is  made,  the  proposal  will  be 

submitted  using  the  factored  pricing. 

As  in  risk  scoring,  individual  vendors'  internal  processes  for  using 
the  technique  vary  with  project  size  and  type.  Proponents  of  this 
type  of  analysis  say  that  it  has  advantages  over  risk  scoring, 
because  the  output  can  be  used  directly  by  proposal  teams  to 
discuss  the  cost  impact  of  risk  with  the  prospect  on  a  task-by-task 
basis. 

c.  Prospect  Assessment 

This  technique  emphasizes  the  prospect's  contribution  to  the  risk 
equation.  Judgments  are  made  (and  may  be  scored)  regarding  the 
prospect's  organization,  systems  skills,  political  commitment  to  the 
effort,  etc.  The  project  is  priced  using  the  vendor's  normal  pricing 
scheme  and  a  factor  is  applied  to  the  margin  to  cover  the  risk. 

This  is  clearly  the  simplest  approach  and  on  the  surface  does  not 
appear  to  be  as  thorough  or  sophisticated  as  the  other  approaches. 
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However,  defenders  of  this  approach  indicate  that  their  normal 
pricing  scheme  accounts  for  the  risk  contribution  inherent  in  the 
project,  making  a  separate  analysis  unnecessary. 

Despite,  the  relatively  methodical  way  vendors  seem  to  approach 
risk  assessment,  they  admit  that  other  overriding  factors 
sometimes  play  a  role  in  deciding  whether  or  not  to  pursue  and 
opportunity. 

•  Most  indicated  that  if  they  could  not  come  up  with  an 
acceptable  price  that  would  yield  a  25-30%  margin  for  large 
fixed-price  projects,  the  deal  probably  was  not  worth  doing. 

•  The  subjective  desirability  of  the  prospect  can  also  have 
considerable  influence.  If  the  prestige  of  having  the  prospect 
as  a  client,  or  the  opportunity  to  work  with  a  client  (that  will 
add  credibility  to  a  vendor's  industry  expertise),  most  vendors 
are  willing  to  accept  higher  risk. 


B  

Pricing  for  Risk  Containment 

Regardless  of  the  approach  used  to  assess  risk,  the  vendor's  price  is 
going  to  reflect  the  assumption  of  risk.  Exhibit  IV-3  shows  the 
proportion  of  contracts  priced  under  fixed-price,  time-and- 
materials,  and  value-based  approaches.  The  averages  and  a  two- 
year  projection  are  shown. 
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EXHIBIT  IV-3 

Proportion  of  Vendor  Projects  Using  Various  Pricing  Approach,  1992  and  1996 

Pricing 
Approach 


Fixed-Price 


Time-and- 
Materials 


Value-based 


Other 


Q  10  20  30  40      ~  50 

Proportion  of  Contracts  (%) 


In  general,  the  proportion  of  usage  in  the  exhibit  is  consistent  with 
the  user  preferences  for  contract  types  states  shown  in  Exhibit 
III-9,  Chapter  III-13.  That  is,  the  relative  use  of  fixed-price,  time- 
and-materials,  and  value-based  contracts  cited  by  vendors,  are  in 
the  same  rank  order  as  user  preferences  for  these  types  of  contracts, 
3.9  for  fixed-price,  3.1  for  time-and-materials,  1.7  for  value  based. 

However,  vendors  anticipate  a  significant  shift  m  the  mix  over  the 
next  two  year  period  with  more  than  an  80%  increase  in  the  usage 
of  value-based  contracts,  largely  at  the  expense  of  27%  decrease  in 
time-and-materials  engagements.  This  shift  is  probably 
attributable  to  a  number  of  influences. 
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•  A  growing  number  of  projects  involving  business  reengineering 
efforts  frequently  identify  large  pools  of  cost  savings  or  profit 
improvements.  In  most  instances,  implementation  will  require 
major  systems  changes  resulting  in  large  systems  integration 
opportunities.  By  linking  some  portion  of  their  profit  margin 
to  a  percentage  of  the  savings  or  improved  profits,  vendors 
accomplish  two  objectives. 

-  Since  the  potential  savings  or  profit  improvement  related  to 
a  major  business  reengineering  effort  is  typically  large,  being 
rewarded  with  even  a  small  percentage  of  the  total  dollar 
benefits  can  create  profit  potential  well  in  excess  of  that 
usually  achieved  on  a  standalone  SI  project. 

-  The  larger  profit  potential  generated  by  value-based 
contracts  provides  a  more  substantial  cushion  to  cover  the 
potential  risk,  and  provides  incentive  for  outstanding 
performance  on  the  part  of  the  project  team. 

Given  the  potential  benefits,  vendors  will  continue  to  push 
for  value-based  pricing  where  business  reengineering  or 
other  situations  create  the  right  opportunity. 

•  Although  fixed-price  contracts  pose  the  highest  risk  to  the 
vendor,  they  clearly  are  the  buyer's  preferred  method  of  doing 
business.  Therefore,  as  indicated  in  Exhibit  IV-3,  it  is  unlikely 
that  vendors  will  be  able  to  reduce  user  interest  in  fixed-price 
contracts.  However,  they  may  be  able  to  convince  users  to 
share  the  financial  gains  associated  with  business 
reengineering-based  projects  by  offering  lower  fixed-prices. 

However,  the  degree  to  which  vendors  can  convert  users  to  value- 
based  pricing  is  still  an  open  question.  Vendors  who  offer  a  full 
spectrum  of  services  such  as  EDS  and  Andersen  Consulting,  claim 
high-levels  of  success  with  this  approach.  Also,  the  growing 
number  of  business  reengineering  efforts  underway  will  create 
additional  opportunities.  However,  with  an  average-interest  level 
in  value-based  pricing  of  1.7  on  a  scale  of  1-5,  users  do  not  appear  to 
be  expressing  much  enthusiasm  for  the  concept  at  the  moment. 

A  number  of  vendors  indicated  they  used  pricing  strategies  other 
than  the  three  major  types  just  discussed.  The  two  methods  most 
frequently  cited  were  phased-fixed  and  range-based  pricing. 
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•  Phase-fixed  is  most  popular  in  situations  where  a  vendor  is 
brought  in  prior  to  the  generation  of  any  detailed 
specifications,  or  when  the  project  will  involve  the  application 
of  leading  edge  technology.  In  these  cases,  there  are  too  many 
unknowns  to  fix  the  price  for  the  entire  engagement  at  the 
start.  Instead,  a  fixed-price  is  established  for  the  first  phase 
and  rough  estimates  for  the  follow-on  phases.  As  one  phase  is 
completed,  fixed-prices  are  established  for  one  or  more  of  the 
following  phases. 

•  A  range-based  pricing  agreement  establishes  a  bandwidth  of 
prices  for  one  or  more  of  the  phases.  This  approach  is  used  in 
many  of  the  same  situations  as  phase-fixed  pricing.  Risk 
conditions  and  their  effect  on  the  price  for  the  work  are 
explicitly  spelled  out  in  the  contract.  At  the  conclusion  of  a 
phase,  or  at  the  end  of  the  contract,  payments  are  made  in 
accordance  with  the  actual  conditions  that  took  place.  From  a 
user's  viewpoint,  this  approach  has  the  advantage  of  putting  a 
ceiling  on  the  price  almost  regardless  of  contingencies. 

These  approaches  tend  to  promote  a  higher  level  of  risk  sharing 
between  vendors  and  users  than  the  standard  fixed-price  approach, 
resulting  in  reduced  vendor  risk,  and  the  cost  to  the  user  as  well. 

Various  combinations  of  all  of  the  pricing  approaches  discussed  in 
this  section  are  being  used  today.  The  analysis  indicates  that 
vendors  will  be  encouraging  (and  users  will  become  more  receptive 
to)  more  sophisticated  approaches  to  pricing,  relying  on 
combinations  of  approaches  to  encourage  greater  risk  sharing  in  the 
future. 


c  

Risk  Control 

As  the  partner  carrying  the  greater  burden  of  risk,  vendors  use  a 
variety  of  strategies  and  techniques  to  insure  that  the  elements  of 
risk  are  managed  once  a  contract  has  been  signed.  This  section 
examines  the  overall  strategies  and  specific  approaches  used  by 
vendors. 
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1.  Key  Strategies  for  Risk  Control 

For  higher  risk  projects,  vendors  have  a  number  of  approaches  they 
apply  throughout  the  project.  Exhibit  IV-4  shows  the  number  of 
times  specific  strategies  were  mentioned  by  survey  respondents. 


EXHIBIT  IV-4 

Vendor  Mechanisms  For  Risk  Management 

Mechanism 


Regular  Reviews 


Special  Processes 


Price  Adjustments 


Personnel 


Other 


0  2  4  6  8 

Number  of  Responses 


Further  explanation  of  each  approach  follows. 

•  The  use  of  regular  reviews  are  a  time-honored  tool  for  project 
control  and  was  the  most  frequently  cited  strategy  mentioned 
by  respondents  for  managing  risk  throughout  a  project.  On 
the  surface,  this  would  seem  like  a  simplistic  approach,  based 
on  the  principle  that  you  "can  expect  what  you  inspect." 
However,  to  deal  with  higher  levels  of  risk  and  the  financial 
implications  of  penalty  clauses  and  performance  bonus 
payments  based  on  milestone  completions,  etc.,  vendors  have 
evolved  much  more  sophisticated  approaches  to  project  reviews 
than  most  internal  IS  departments. 
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-  Independent  audit  groups  or  quality  control  organizations 
are  frequently  used  rather  than  relying  on  status  reporting 
from  the  SI  team  itself  as  the  sole  indicator  of  the  health  of 
the  project.  These  outside  organizations  frequently  have 
authority  to  mandate  specific  actions  to  realign  projects  that 
are  off  track. 

-  Review  cycles  are  usually  adjusted  on  the  basis  of  identified 
problems;  e.g.,  a  project  where  corrective  action  is  prescribed 
will  certainly  undergo  more  frequent  scrutiny  than  one  that 
is  meeting  schedule  and  milestone  commitments. 

-  Special  review  processes  and  shorter  review  cycles  may  be 
set  up  to  deal  with  high-risk  project  tasks. 

-  Conservative  reporting  of  accomplishment  is  used  by  some 
vendors  to  insure  that  project  teams  are  focused  on  upcoming 
milestones.  Under  this  approach  the  project  team  is  only 
given  credit  at  a  review  for  milestones  actually  completed. 
No  credit  is  given  or  reported  to  the  customer  for  partially 
completed  tasks.  The  review  focuses  on  the  status  of  the 
work  in  process  and  obtaining  commitment  to,  or  resolving 
the  problems  associated  with,  meeting  the  upcoming 
milestones. 

•  Special  processes  or  systems,  the  second  most  frequently  cited 
risk  containment  technique,  are  also  widely  used. 

-  A  risk  reassessment  process  may  bo  used  prior  to  the 
initiation  of  each  new  phase  of  a  project  to  re-evaluate  the 
risk  and  adjust  the  containment  strategy  based  project 
experience  to  date. 

-  Continuous  or  on-line  problem  resolution  systems  are  also 
growing  in  popularity.  This  approach  involves  networking 
the  project  team  into  experience  databases  and  the  vendor's 
subject  experts  using  a  technology  such  as  Lotus  NOTES. 
Project  team  members  utilize  these  resources  on  an  on-going 
basis  to  gain  access  to  the  best  thinking  and  experience  that 
the  vendor  has  on  handling  technical  and  management 
problems  as  they  arise. 
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•  Price  adjustments  are  also  a  popular  mechanism  with  vendors. 
Although,  in  and  of  themselves,  they  certainly  do  not 
contribute  to  risk  containment,  they  do  reduce  the  risk  of 
financial  loss.  Obviously,  the  terms  and  conditions  under 
which  adjustments  can  be  made  must  be  carefully  negotiated 
at  the  front  end  of  the  project.  The  phase-fixed  and  range- 
based  pricing  strategies  discussed  earlier  are  two  methods  of 
building  the  ability  to  adjust  prices  into  the  contract. 

•  The  selection  and  assignment  oi personnel  as  a  risk 
containment  strategy  was  only  mentioned  as  a  major  strategy 
by  three  of  the  vendors.  However,  every  vendor  pointed  out 
the  importance  of  staffing  somewhere  during  their  interview. 
Populating  new  projects  with  people  whose  experience  and 
skills  match  the  task  at  hand  is  essential.  Furthermore,  some 
vendors  use  highly  formalized  processes  to  make  the 
selections. 

-  The  use  of  experience  and  skills  databases  is  not  uncommon, 
some  even  dealing  with  personality  profiles.  . 

-  The  assignment  of  subject-area  specialists  to  high-risk  tasks 
is  almost  universal. 

Both  vendors  in  the  "other"  category  indicated  that  although  they 
had  internal  strategies  and  processes,  they  felt  that  the  most 
effective  way  to  contain  or  manage  risk  was  proactive  inclusion  of 
the  customer  in  the  risk  assessment  and  management  aspects  of 
the  project.  From  their  viewpoint,  this  ongoing  involvement  helped 
set  customer  expectations  and  frequently  yielded  user  suggested 
approaches  to  risk  management  that  resulted  in  more  equitable 
risk  sharing. 

2.  Task  Specific  Risk  Control  Mechanisms 

In  addition  to  the  broad-based  strategies  discussed  above,  vendors 
have  a  wide  variety  of  techniques,  models  and  tools  they  employ  to 
deal  with  risk  in  the  specific  aspects  of  an  engagement  such  as: 

•  Estimating 

•  Project  Management 

•  Subcontractor  Management 
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•  Project  Pricing 

a.  Estimating 

Vendors  have  invested  considerable  effort  in  developing  and 
refining  estimating  techniques.  Competitive  bidding  and  the 
frequent  requirement  to  commit  to  fixed-prices  provides  a  strong 
incentive  to  have  estimates  be  as  accurate  as  possible. 
Furthermore,  a  major  error  during  this  phase  will  make  a  project 
unprofitable  regardless  of  how  well  it  is  executed. 

Most  vendors  have  evolved  a  proprietary  methodology  to  deal  with 
estimating.  However,  there  appears  to  be  a  number  of  components 
that  these  methodologies  have  in  common. 

•  The  use  of  function  point  analysis  to  establish  general  resource 
requirements  and  overall  scope  was  mentioned  by  eight  of  the 
eleven  respondents. 

•  Independent  of  (yet  in  addition  to)  function  point  analysis, 
most  firms  have  established  costing  standards  to  price  projects 
on  a  task-by-task  basis. 

•  Virtually  all  the  proprietary  methodologies  use  models, 
frequently  based  on  experienced  database,  to  factor  projects  for 
risk.  In  addition  to  dealing  with  technical  complexity,  etc., 
many  of  the  models  also  account  for  the  experience  levels  of 
assigned  personnel.  Some  incorporate  components  of  CASE 
methodologies  such  as  lEF. 

A  technique  used  by  many  firms  is  to  apply  multiple  estimating 
methods  to  the  same  project,  frequently  using  different  estimating 
teams.  Results  are  compared  and  where  there  are  significant 
discrepancies  between  estimates  for  a  given  task  additional 
analysis  is  performed  to  arrive  at  a  reconciled  estimate. 

Although  not  as  common  as  the  just  mentioned  approaches,  monte 
carlo  simulation  was  mentioned  by  a  few  respondents  as  a  means  of 
providing  a  probability  distribution  of  project  costs. 

b.  Project  Management 

Every  vendor  has  its  own  approach  to  project  management,  but  as 
in  the  case  of  risk  management  processes,  there  are  some  common 
themes. 


©  1994  by  INPUT  Reproduction  Prohibited 


IV-15 


CONTRACTUAL  APPROACHES  TO  PROJECT  RISK  REDUCTION 


INPUT 


•  All  involve  milestone  tracking  and  regular  reviews,  and  the 
direction  appears  to  moving  toward  automated  systems  that 
facilitate  this  activity  through  continuous  monitoring  of 
resource  usage,  etc. 

•  Most  have  built-in  problem  identification  and  escalation 
processes  to  catch  potential  problems  and  resolve  them  at  the 
earliest  possible  stage. 

•  Although  generally  proprietary,  many  incorporate  tools  such 
as  the  Project  Management  Workbench  and  Microsoft  Project. 

A  number  of  vendors  have  developed  formal  quality  assurance 
processes  that  rely  on  specialists  to  monitor  project  processes  and 
assess  the  quality  of  deliverables  as  they  are  produced.  Some 
incorporate  users  directly  into  the  quality  team,  encouraging  a 
partnership  approach  to  the  project  management  process. 

c.  Subcontracting 

Most  vendors  do  not  believe  that  the  use  of  subcontractors 
contributes  a  significant  element  of  risk  to  most  projects. 
Prequalifying  subcontractors  and  repeated  use  of  preferred 
suppliers  are  the  most  frequently  used  techniques  to  keep  whatever 
risks  do  exist  to  a  minimum. 

Other  approaches  to  minimizing  risk  in  this  area  include: 

•  Full  integration  of  the  subcontractor's  personnel  into  the 
vendor's  project  team.  In  many  instances  integration  starts  at 
the  estimating  phase. 

•  Consistent  alignment  of  the  performance  and  payment  terms 
of  the  subcontractor's  contract  with  the  terms  and  conditions  of 
payment  that  will  apply  to  the  vendor. 

d.  Pricing 

Vendors  indicate  that  the  best  protection  against  making  an  error 
in  pricing  is  to  make  sure  that  solid  resource  estimating  techniques 
are  in  place.  The  approach  must  break  the  project  down  into 
specific  tasks,  and  explicitly  account  for  risk.  In  other  words, 
insure  that  the  costing  has  been  done  properly.  Assuming  the 
project  gets  estimated  properly,  vendors  will  promote  incentive 
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mechanisms,  range-based  and  phase-fixed  contract  mechanisms 
wherever  possible  to  reduce  the  exposure  to  financial  loss  resulting 
from  unforeseen  events. 

Exhibit  IV-5  summarizes  the  techniques  used  by  vendors  to  mange 
risk  in  key  SI  management  processes,  and  gives  INPUT'S  account  of 
the  degree  to  which  each  technique  is  currently  used. 


Techniques  Used  by  Vendors  to  Minimize  Risk  in 
Key  SI  Management  Processes 


Process 


Tool  or  Technique 


Level  of  Use 


Estimating 


Function  Point  Analysis 
Proprietary  Methodologies 
Multiple  Estimates 
Monte  Carlo  Techniques 


Project  Management 


Continuous  Tracking  and  Review 
Proprietary  Project  Mgt.  Process 
Quality  Assurance  System 


Subcontracting 


Prequalified/Preferred  Suppliers 
Full  Integration  into  Team 
Payment  or  Contract  Mechanisms 


Pricing 


Task  Estimating  Including  Risk 
Incentive  Mechanisms 
Range-based/Phase-fixed 


=  Heavy  Use 
=  Medium  Use 
=  Light  Use 
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Summary 

In  systems  integration  projects,  vendors  generally  assume  the  bulk 
of  the  risk.  Consequently,  they  have  invested  heavily  in  processes 
to  identify  sources  of  risk,  approaches  to  measuring  its  impact  on 
costs  and  methods  to  manage  or  contain  it. 

In  general,  a  vendor's  management  of  risks  takes  two  paths: 

•  Insuring  that  key  processes  and  activities,  such  as  estimating, 
project  management,  subcontractor  selection  and  management 
and  project  pricing,  explicitly  recognize  and  account  for  risk 

•  Incorporating,  to  the  degree  possible,  contract  mechanisms  ' 
that  encourage  risk  sharing  with  the  buyer  and  allow  for 
adjustments  to  fixed-price  deals  based  on  contingencies 

To  accomplish  the  former,  vendors  have  invested  heavily  in 
developing  proprietary  processes  to  deal  with  risk  from  the  proposal 
stage  through  delivery.  These  processes  frequently  incorporate  off- 
the-shelf  technologies  or  standard  offerings  such  as  nonproprietary 
CASE  tools.  However,  it  is  probably  safe  to  assume  that  each 
vendor's  system  for  risk  identification  and  management  is  unique. 

From  a  vendor's  perspective  the  fixed-price  contracts  represent  the 
highest  risk  situation — and  conversely,  time-and-materials 
agreements  present  the  least  exposure.  To  strike  a  balance 
between  these  positions,  vendors  are  encouraging  the  use  of  newer 
and  more  innovative  pricing  strategies,  including  phase-fixed  and 
range-based  approaches  to  promote  risk  sharing  and  reduce 
financial  exposure. 
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Risk  in  Contracts 


How  risk  is  managed  from  a  process  and  contractual  point  of  view, 
clearly  has  impact  on  the  relationship  between  users  and  vendors. 
While  users  and  vendors  like  to  talk  about  systems  integration, 
projects  as  partnerships,  contract  mechanisms  employed  to  contain 
risk  can  have  a  negative  effect  on  the  partnership  concept. 

This  chapter  discusses  the  impact  of  risk  management  strategies  on 
the  relationship  and  discusses  contract  trends  that  may  offer  some 
solutions  to  the  problems  that  current  contract  mechanisms  can 
have  on  effective  user/vendor  relationships. 


A  

UserA/endor  Relationships 

By  and  large,  the  vast  majority  of  systems  integration  projects 
prove  to  be  successful.  The  partnership  works  and  client 
satisfaction  is  generally  high. 

Still,  vendors  admit  that  although  almost  all  projects  go  to 
completion,  there  is  some  percentage  that  fails  to  meet  the  client's 
originally  stated  business  needs.  Likewise,  there  is  a  percentage 
that  results  in  a  financial  loss  to  the  vendor. 

Exhibit  V-1  shows  how  the  vendors  estimated  those  percentages  by 
number  of  projects  and  dollar  volume. 
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Summary 

In  systems  integration  projects,  vendors  generally  assume  the  bulk 
of  the  risk.  Consequently,  they  have  invested  heavily  in  processes 
to  identify  sources  of  risk,  approaches  to  measuring  its  impact  on 
costs  and  methods  to  manage  or  contain  it. 

In  general,  a  vendor's  management  of  risks  takes  two  paths: 

•  Insuring  that  key  processes  and  activities,  such  as  estimating, 
project  management,  subcontractor  selection  and  management 
and  project  pricing,  explicitly  recognize  and  account  for  risk 

•  Incorporating,  to  the  degree  possible,  contract  mechanisms  " 
that  encourage  risk  sharing  with  the  buyer  and  allow  for 
adjustments  to  fixed-price  deals  based  on  contingencies 

To  accomplish  the  former,  vendors  have  invested  heavily  in 
developing  proprietary  processes  to  deal  with  risk  from  the  proposal 
stage  through  delivery.  These  processes  frequently  incorporate  off- 
the-shelf  technologies  or  standard  offerings  such  as  nonproprietary 
CASE  tools.  However,  it  is  probably  safe  to  assume  that  each 
vendor's  system  for  risk  identification  and  management  is  unique. 

From  a  vendor's  perspective  the  fixed-price  contracts  represent  the 
highest  risk  situation — and  conversely,  time-and-materials 
agreements  present  the  least  exposure.  To  strike  a  balance 
between  these  positions,  vendors  are  encouraging  the  use  of  newer 
and  more  innovative  pricing  strategies,  including  phase-fixed  and 
range-based  approaches  to  promote  risk  sharing  and  reduce 
financial  exposure. 
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Risk  in  Contracts 


How  risk  is  managed  from  a  process  and  contractual  point  of  view, 
clearly  has  impact  on  the  relationship  between  users  and  vendors. 
While  users  and  vendors  like  to  talk  about  systems  integration, 
projects  as  partnerships,  contract  mechanisms  employed  to  contain 
risk  can  have  a  negative  effect  on  the  partnership  concept. 

This  chapter  discusses  the  impact  of  risk  management  strategies  on 
the  relationship  and  discusses  contract  trends  that  may  offer  some 
solutions  to  the  problems  that  current  contract  mechanisms  can 
have  on  effective  user/vendor  relationships. 


A  

User/Vendor  Relationships 

By  and  large,  the  vast  majority  of  systems  integration  projects 
prove  to  be  successful.  The  partnership  works  and  client 
satisfaction  is  generally  high. 

Still,  vendors  admit  that  although  almost  all  projects  go  to 
completion,  there  is  some  percentage  that  fails  to  meet  the  client's 
originally  stated  business  needs.  Likewise,  there  is  a  percentage 
that  results  in  a  financial  loss  to  the  vendor. 

Exhibit  V-1  shows  how  the  vendors  estimated  those  percentages  by 
number  of  projects  and  dollar  volume. 
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EXHIBIT  V-1 


Proportion  of  Contracts  Failing  to  Meet  User  Needs 
and/or  Representing  Losses  to  Vendors 


Category 


Vendor  Loses 


Failed  User 
Expectations 


14 


□  By  $  Volume 

□  By  Number 


11 


4         6         8        10        12  14 
Proportion  of  Contracts  (%) 


The  data  indicates  that  there  is  a  one  in  ten  chance  that  the  project 
will  fail  to  meet  user  expectations  and/or  lose  money  for  the  vendor. 
In  fact,  several  of  the  vendors  commented  that  they  have  just  about 
as  many  failures  of  each  type  (failing  to  meet  user  expectations,  or 
losing  money)  on  large  and  small  projects.  In  either  case,  users 
tend  to  place  the  blame  on  vendors,  and  vendors,  who  by  contract 
usually  assume  the  bulk  of  the  responsibility  for  delivering 
acceptable  results,  tend  to  agree,  indicating  that: 

•  In  situations  where  user  expectations  were  not  met,  the 
problem  is  most  likely  been  with  their  management  of  the 
early  phases  of  the  project 

•  When  they  lose  money  they  have  usually  underestimated  the 
risks. 
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However,  vendors  also  point  out  that  the  terms  and  conditions  of 
the  contract  can  make  the  difference  between  a  successful  and  a 
merely  tolerable  working  arrangement,  as  well  as  the  probability  of 
success  of  the  project. 

•  Most  vendors  and  about  25%  of  the  users  felt  that  penalty 
clauses  worked  against  the  partnership  concept.  In  fact,  when 
enforced  to  the  "letter  of  the  law"  they  can  cause  major 
disruptions  and  refocus  energy  that  should  be  spent  on  joint 
problem  solving  to  various  kinds  of  defensive  maneuvering. 

•  Liquidated  damage  clauses  are,  for  the  most  part,  considered 
simply  unacceptable.  Over  half  of  the  study's  vendor 
participants  indicated  that  they  simply  would  not  sign  an 
agreement  that  called  for  liquidated  damages. 

•  Processes  prescribed  by  contract  which  focus  on  pinpointing 
blame  rather  than  problem  identification  and  resolution,  fall 
into  the  same  category. 

Obviously,  most  vendors  favor  contract  mechanisms  that  encourage 
some  level  of  risk  sharing  and  joint  problem  solving.  They  are 
willing  to  lower  base  prices,  particularly  on  fixed-price  proposals,  if 
the  opportunity  exists  to  share  in  a  project's  financial  benefits 
through  value-based  pricing  or  other  incentive  mechanisms. 

In  general,  the  more  the  project  feels  like  a  joint  venture,  the  better 
the  relationship  is  likely  to  be — and  from  the  vendors  perspective, 
the  higher  the  probability  of  success. 


B  

Trends  in  Risk  Management 


The  most  dominant  trend  in  risk  management  today  is  a  movement 
toward  increased  risk  sharing  in  systems  integration  engagements. 
This  movement  is  impacting  both  the  nature  of  the  contracts  and 
the  management  of  systems  integration  projects. 

•  Vendors  are  promoting  more  risk  sharing  in  the  belief  that  it 
will  increase  the  probability  of  success  as  well  as  improve 
profits  over  the  long  run. 
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•  Users  are  becoming  more  receptive  to  the  increasingly 
sophisticated  pricing  and  contracting  approaches  required  to 
accompHsh  risk  sharing. 

•  Improved  processes  and  technology  are  providing  the 
information  necessary  to  identify,  assess  and  assist  in  the 
process  of  managing  risk. 

These  underlying  forces  are  impacting  the  nature  of  the  contract 
arrangements  between  users  and  vendors  and  the  management 
processes  used  throughout  an  engagement.  ' 

1.  Contract  Trends 

The  major  trends  impacting  contracts  are: 

•  An  accelerating  shift  from  time-and-materials  pricing  to  value- 
based  or  other  incentive  based  approaches 

•  A  movement  toward  pricing  schemes  such  as  range-based  and 
phaso-fixed  which  encourage  risk  sharing  (with  or  without 
incentive  clauses)  and  acknowledge  at  the  start  of  a  project 
that  there  may  be  elements  of  risk  that  simply  cannot  be 
properly  estimated  in  financial  terms. 

•  A  growing  tendency  to  include  detailed  contract  specifications 
for  user  resource  requirements  down  to  the  level  of  phase  and 
task. 

•  The  use  of  joint  venture  development  efforts  between  buyers 
and  vendors  to  deal  with  extremely  high-risk  projects 
involving  advanced  or  unproved  technology. 

Exhibit  V-2  lists  the  major  trends  in  SI  contracts  and  gives 
input's  assessment  of  their  potential  benefits  and  impacts. 


V-4 


1994  by  INPUT  Reproduction  Prohibited. 


BIRR 


CONTRACTUAL  APPROACHES  TO  PROJECT  RISK  REDUCTION 


INPUT 


Trends  in  Systems  Integration  Contracts 




Trend 

Impact/Benefit 

Shift  to  Value-based  and 
Incentive  Pricing 

•  Increased  incentive  for  integrators  to  apply 
innovative  approaches 

•  Improved  partnership  relationship 

•  Lower  user  costs  to  cover  risk 

Movement  Toward  Range- 
based  and  Phase-fixed  Pricing 

•  Objective  recognition  of  the  inability  to 
define  certain  elements  of  risk 

•  Lower  costs  to  user  and  an  inducement  to 
user  participation  to  the  partnership 

Contractual  Commitment  to 
User  Involvement 

•  Insures  user  resources  will  be  available  to 
meet  contract  commitments 

•  Increases  sense  of  partnership  and 
participation 

Join  Venture  for  Leading  Edge 
Efforts 

•  Formalizes  the  concept  of  risk  sharing  with 
shared  benefits 

•  Promotes  user  involvement  in  the  design 
process 

2.  Process  Trends 

The  tendency  toward  increased  risk  sharing  is  also  impacting  the 
nature  of,  and  user  involvement  in,  the  management  processes  used 
for  implementation. 

•  An  increase  in  the  use  of  computer-assisted  continuous 
monitoring  processes  supported  by  on-line  computer 
applications. 

•  A  growth  in  the  use  of  specialized  quality  assurance 
assessment  teams  to  provide  early  problem  identification  and 
recommended  solutions. 
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•  An  increase  in  the  inclusion  of  user  personnel  in  quality 
assurance,  ongoing  risk  assessment  and  other  project 
monitoring  and  control  processes,  formerly  considered  internal 
to  the  vendor. 

•  A  growing  use  of  prototyping  and  application  modeling  to 
ensure  user  satisfaction  with  the  end  product. 

Exhibit  V-3  lists  the  major  trends  in  the  processes  that  impact  risk 
management  of  SI  engagements,  including  INPUT'S  assessment  of 
their  benefits.  ' 


EXHIBIT  V-3 

Trends  in  Systems  Integration  Project  Management  Processes 


Trend 

Impact/Benefit 

Computer-assisted  Monitoring 
Process 

•  Early  identification  of  problems,  and 
access  to  expertise  for  resolution 

•  Integration  of  user  into  the  monitoring 
and  management  process 

•  Provision  of  the  data  necessary  for 
management  of  more  sophisticated 
contract  schemes 

Commitment  to  Formal  QA 
Processes 

•  More  objective  evaluation  of  current 
status  and  suggested  changes 

•  Ongoing  monitoring  of  quality  through 
the  use  of  computer-based  tools 

Inclusion  of  User  Personnel  in 
Vendor-managed  Processes 

•  Promotes  the  partnership  concept  by 
direct  participation  in  an  open  setting 

•  Provides  more  direct  user  feedback 
than  formal  status  reviews,  etc. 

Prototyping  and  Application 
Modeling 

•  Provides  cost  effective  method  of 
testing  the  quality  and  reality  of 
specifications  early  in  the  process 

•  Helps  set  user  expectations  for 
functionality  of  the  delivered  system 

As  these  trends  continue  to  evolve,  they  are  likely  to  be  reinforced 
by  the  fact  that  there  is  a  growing  community  of  users  who  have 
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been  through  multiple  SI  engagements.  As  user  comfort  levels  rise 
with  the  use  of  outside  SI  services,  receptivity  to  more  innovative 
approaches  to  contracting  and  management  processes  that  promote 
risk  sharing  is  likely  to  grow  as  well.  The  opportunity  to  reduce 
costs  by  risk  sharing  will  provide  an  additional  incentive. 

c  

Longer  Range  Directions 

Clearly  most  vendors  would  rather  have  a  long-term  relationship 
with  what  they  consider  to  be  quality  customers. 

•  Risk  is  reduced  in  long-term  relationships  simply  because  of 
the  common  understanding  generated  by  the  relationship. 

•  The  cost  of  business  to  the  vendor,  as  well  as  the  potential 
cost  to  the  buyer,  is  reduced  due  to  the  fact  that  vendor 
marketing  expenses  to  a  long-term  customer  are  significantly 
lower  than  first-time  prospects. 

In  support  of  this  concept  is  the  fact  that  at  least  two  of  the  vendors 
in  the  survey  considered  the  potential  for  a  long-term  relationship 
the  most  significant  factor  in  evaluating  the  desn-ability  of  potential 
SI  prospects. 

As  a  consequence,  long-term  relationships,  evolving  for  all  practical 
purposes  into  systems  operations  applications  management 
agreements,  probably  represent  the  wave  of  the  future.  As  these 
agreements  are  developed  it  is  likely  that  the  actual  contract 
between  supplier  and  buyer  will  decrease  in  importance  in  terms  of 
day-to-day  management  of  risk  and  vendor  compensation.  Instead, 
they  will  specify  the  processes  by  which  risk  and  other  issues  are 
managed,  providing  a  framework  for  the  relationship  rather  than  a 
blueprint  for  a  specific  engagement. 
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Definition  of  Terms 


A   .  

Introduction 

input's  Definition  of  Terms  provides  the  framework  for  all  of 
input's  market  analyses  and  forecasts  of  the  information 
services  industry.  It  is  used  for  all  U.S.  programs.  The  structure 
defined  in  Exhibit  A-1  is  also  used  in  Europe  and  for  the 
worldwide  forecast. 

One  of  the  strengths  of  INPUT'S  market  analysis  services  is  the 
consistency  of  the  underl3dng  market  sizing  and  forecast  data. 
Each  year  INPUT  reviews  its  industry  structure  and  makes 
changes  if  they  are  required.  When  changes  are  made  they  are 
carefully  documented  and  the  new  definitions  and  forecasts 
reconciled  to  the  prior  definitions  and  forecasts.  INPUT  clients 
have  the  benefit  of  being  able  to  track  market  forecast  data  from 
year  to  year  against  a  proven  and  consistent  foundation  of 
definitions. 
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Exhibit  A-1 
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B  ^  

Overall  Definitions  and  Analytical  Framework 

1.  Information  Services 

Information  Services  are  computer/telecommunications-related 
products  and  services  that  are  oriented  toward  the  development  or 
use  of  information  systems.  Information  services  t3rpically  involve 
one  or  more  of  the  following: 

•  Use  of  vendor-provided  computer  processing  services  to  develop 
or  run  applications  or  provide  services  such  as  disaster 
recovery  or  data  entry  (called  Processing  Services) 

•  A  combination  of  computer  equipment,  packaged  software  and 
associated  support  services  which  will  meet  an  application 
systems  need  (called  Turnkey  Systems) 

•  Packaged  software  products,  including  systems  software  or 
applications  software  products  (called  Software  Products) 

•  People  services  that  support  users  in  developing  and  operating 
their  own  information  systems  (called  Professional  Services) 

•  The  combination  of  products  (software  and  equipment)  and 
services  where  the  vendor  assumes  total  responsibility  for  the 
development  of  a  custom  integrated  solution  to  an  information 
systems  need  (called  Systems  Integration) 

•  Services  that  provide  operation  and  management  of  all  or  a 
significant  part  of  a  user's  information  systems  functions 
under  a  long-term  contract  (called  Systems  Operations) 

•  Services  that  support  the  delivery  of  information  in  electronic 
form — typically  network-oriented  services  such  as  value-added 
networks,  electronic  mail  and  document  interchange  (called 
Network  Applications) 

•  Services  that  support  the  access  and  use  of  public  and 
proprietary  information  such  as  on-line  databases  and  news 
services  (called  Electronic  Information  Services) 

•  Services  that  support  the  operation  of  computer  and  digital 
communication  equipment  (called  Equipment  Services) 
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In  general,  the  market  for  information  services  does  not  involve 
providing  equipment  to  users.  The  exception  is  where  the 
equipment  is  part  of  an  overall  service  offering  such  as  a  turnkey 
system,  a  systems  operations  contract  or  a  systems  integration 
project. 

2.  Systems  Integration  (SI) 

Systems  integration  is  a  vendor  service  that  provides  a  complete 
solution  to  an  information  system,  networking  or  automation 
development  requirement  through  the  custom  selection  and 
implementation  of  a  variety  of  information  system  products  and 
services.  A  systems  integrator  is  responsible  for  the  overall 
management  of  a  systems  integration  contract  and  is  the  single 
point  of  contact  and  responsibility  to  the  buyer  for  the  delivery  of 
the  specified  system  function,  on  schedule  and  at  the  contracted 
price.  (Refer  to  Exhibit  A-2.) 

The  components  of  a  systems  integration  project  are  the  following: 

•  Equipment  -  information  processing  and  communications 
equipment  required  to  build  the  systems  solution.  This 
component  may  include  custom  as  well  as  off-the-shelf 
equipment  to  meet  the  unique  needs  of  the  project.  The 
systems  integration  equipment  category  excludes  turnkey 
systems  by  definition. 

•  Software  products  -  prepackaged  applications  and  systems 
software  products. 

•  Professional  services  -  the  value-added  component  that  adapts 
the  equipment  and  develops,  assembles,  or  modifies  the 
software  and  hardware  to  meet  the  system's  requirements.  It 
includes  all  of  the  professional  services  activities  required  to 
develop,  implement,  and  if  included  in  the  contract,  operate  an 
information  system,  including  consulting,  program/project 
management,  design  and  integration,  software  development, 
education  and  training,  documentation,  and  systems 
operations  and  maintenance. 
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Products/Services  in 
Systems  Integration  Projects 

Equipment 

•  Information  systems 

•  Communications 

Software  Products 

•  Systems  software 

•  Applications  software 

Professional  Services 

•  Consulting 

-  Feasibility  and  trade-off  studies 

-  Selection  of  equipment,  network  and  software 

•  Program/project  management 

•  Design/integration 

-  Systems  design 

-  Installation  of  equipment,  network,  and  software 

-  Demonstration  and  testing 

•  Software  development 

-  Modification  of  software  packages 

-  Modification  of  existing  software 

-  Custom  development  of  software 

•  Education/training  and  documentation 

•  Systems  operations/maintenance 

Other  Miscellaneous  Products/Services 

•  Site  preparation 

•  Data  processing  supplies 

•  Processing/network  services 

•  Data/voice  communication  services 
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•  Other  services  -  most  systems  integration  contracts  include 
other  services  and  product  expenditures  that  are  not  classified 
elsewhere.  This  category  includes  miscellaneous  items  such 
as  engineering  services,  automation  equipment,  computer 
supplies,  business  support  services  and  supplies,  and  other 
items  required  for  a  smooth  development  effort. 

3.  Professional  Services 

This  category  includes  four  submodes:  consulting,  education  and 
training,  software  development,  and  applications  management. 
Exhibit  A-3  provides  additional  detail. 

•  Consulting:  Services  include  management  consulting  (related 
to  information  systems),  information  systems  re-engineering, 
information  systems  consulting,  feasibility  analysis  and  cost- 
effectiveness  studies,  and  project  management  assistance. 
Services  may  be  related  to  any  aspect  of  the  information 
system,  including  equipment,  software,  networks  and  systems 
operations. 

•  Education  and  Training:  Services  that  provide  training  and 
education  or  the  development  of  training  materials  related  to 
information  systems  and  services  for  the  information  systems 
professional  and  the  user,  including  computer-aided 
instruction,  computer-based  education,  and  vendor  instruction 
of  user  personnel  in  operations,  design,  programming,  and 

documentation.  Education  and  training  provided  by  school 
systems  is  not  included.  General  education  and  training 
products  are  included  as  a  cross-industry  market  sector. 

•  Software  Development:  Services  include  user  requirements 
definition,  systems  design,  contract  programming, 
documentation,  and  implementation  of  software  performed  on 
a  custom  basis.  Conversion  and  maintenance  services  are  also 
included. 
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Exhibit  A-3 


Professional  Services  Market  Structure 


Business 
Process 
Reengineering 


Professional  Services 


Software 
Development/ 
Maintenance 


User  Requirements 
Definition 

Systems  Design 

System  Conversion 

Data  Base 
Design 

Programming 

Testing 

System  Modification 

Documentation/ 
Tecfinical  Writing 

Network 
Development 

Other 


Installation 
Planning 

Network  Planning 
and  Design 

Information 
Systems  Audit 

IS  Planning 

IS  Security/Audit 

System  Evaluation 

IS  Personnel  Planning 

Systems  Analysis 

IS  Policies  and 

Procedures 

Development 

Project  Management 

Other 


Education  & 
Training* 


Computer 

Operations 

Training 

IS  Management 
Training 

Analyst/ 

Programmer 

Training 

Systems  Use 
Training 

Video  Instruction 
Other 


*Related  to  computer  systems,  topics,  or  issues 
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User  Interview  Guide 


I  am  calling  from  INPUT,  an  international  research  and  consulting  company.  We  are 
conducting  a  study  on  customers'  approaches  to  estimating  and  managing  risk  in  SI 
projects  and  the  associated  impact  on  contracts  between  SI  vendors  and  their 
customers.  The  information  that  you  provide  will  be  confidential  and  neither  your 
name  nor  your  company's  name  will  be  connected  with  any  information  in  the  study. 
In  return  for  your  assistance,  we  will  provide  you  with  a  summary  of  our  findings  at  no 
charge. 


A  

Management  of  Risk 

There  are  many  potential  areas  of  risk  and  uncertainty  in  bidding  for  and  performing 
projects.  Examples  include:  estimation  of  costs,  definition  of  schedules,  availability  of 
skills  etc.  I  would  like  to  start  by  asking  some  questions  regarding  the  identification 
and  management  of  risk. 

1.  On  a  scale  of  1-5,  where  1  =  negligable  and  5  =  very  significant,  how  much  risk 
would  you  say  there  is  in  each  of  the  following  areas  : 

Factor  Rating 

Initial  identification  of  requirements  by  users   

Lack  of  user  involvement  during  course  of  project   

Project  management  by  SI  vendor's  project  manager   

Project  management  your  project  manager   
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Factor  Rating 

Resource  estimating  by  the  vendor   

Resource  estimating  by  your  organization   

Risk  evaluation  by  vendor  at  start  of  the  project   

Risk  evaluation  by  users  at  start  of  the  project   

Control  over  end-user  requests  for  change   

Other  (please  specify)    

2.  Which  of  the  following  steps  do  you  take  to  minimize  the  risk  in  projects? 
Automated  planning  of  resource  requirements  (  Y  /  N  ) 
Computer-based  project  management  (  Y  /  N  ) 
Formalized  change  control  procedures  (  Y  /  N  ) 
Dedicated  user  representatives  on  the  project  team       (  Y  /  N  ) 

Other  (please  specify)  ^  Y 

3.  What  contractual  mechanisms  do  you  include  to  minimize  risk  in  projects  ?  Also, 
are  there  any  you  do  not  use  now  but  may  use  in  the  next  2  years? 


Currently  Use 


Future  Use 


Project  milestones  included  in  contracts 


(  Y  /  N  ) 


( Y/N) 


Performance  clauses 


(Y/N) 


(Y/N) 


Guaranties  or  warranties 


(Y/N) 


(  Y  /  N  ) 


Bonus  payments  for  early  completion/ 
costs  less  than  estimates,  etc. 


(  Y  /  N  ) 


(  Y  /  N  ) 


Others  (please  specify) 


(Y) 


(Y) 
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Working  with  an  SI  Vendor 

4.  Please  rate  on  a  scale  of  1-5,  (where  1  =  not  at  all  desirable  and  5  =  extremely 
desirable),  how  desirable  is  it  for  the  following  responsibilities  to  be  taken  by  your 
organization  and/or  by  the  vendor  to  ensure  successful  information  systems 
projects: 

User  Vendor 

  Take  prime  reponsibility  for  the  complete   

project 

  Take  prime  reponsibility  for  selected  elements   

of  the  project 

  Supply  supporting  services  as  required  without   

responsibility  for  the  entire  system 

Other  (please  specify) 


5.  Please,  rate  each  method  on  a  scale  of  1-5,  (where  1  =  not  at  all  desirable  and 
5  =  extremely  desirable),  how  desirable  is  it  for  an  external  vendor  to  take 
responsibility  for  each  of  the  following  types  of  activity: 

Activity  Rating 

Business  process  reengineering   

Functional  specification   

Detailed  systems  design   

System  development  '   

System  implementation   

Training  of  users  2  

System  performance  measurement   

System  tuning   
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Activity  Rating 

Conformance  to  business  needs   

Cost  of  system   

6.  On  a  scale  of  1-5,  (where  1  =  extremely  poor  and  5  =  excellent),  how  would  you  rate 
the  quality  of  partnership  between  your  organization  and  SI  vendors  you  have 
used? 

Rating  =  


7.  Please  rate  on  a  scale  of  1-5,  (where  1  =  not  at  all  desirable  and  5  =  extremely 

desirable),  how  desirable  is  it  to  improve  the  quality  of  partnership  between  vendor 
and  client  ? 

Rating  =  


8.  On  a  scale  of  1-5,  (where  1  =  not  important  and  5  =  extremely  important),  how 
would  you  rate  the  following  as  contributors  towards  partnership  success  ?  Do  you 
think  current  partnerships  need  improvement  in  each  of  these  areas  ? 

Rating  of  .  Requires 
Importance  Improvement 

Common  culture    (  Y  /  N  ) 

Willingness  of  vendor  to 

assume  risk  for  the  project    (  Y  /  N  ) 

Vendor  understanding  of 

your  business    (  Y  /  N  ) 


Other  (please  specify) 


( Y/N) 


9.  Do  you  feel  the  contract  between  the  vendor  and  your  organization  could  be  changed 
to  encourage  improved  partnerships  by  inclusion  of  the  following? 

Project  milestones  (  Y  /  N  ) 

Performance  clauses  (  Y  /  N  ) 

Guaranties  or  warranties  (  Y  /  N  ) 
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Bonus  payments  for  early  completion/ 
costs  less  than  estimates  etc 


( Y/N) 


Others  (please  specify) 


(Y) 


C 


Pricing  Mechanisms 

10.  Rating  on  a  scale  of  1-5,  (where  1  =  not  at  all  and  5  =  very  favourably),  to  what 
extent  do  you  use  each  of  the  following  pricing  mechanisms  when  subcontracting 
information  systems  projects  ? 


Fixed-price 

Time-and-materials 

Value-based 

(i.e.  the  SI  vendor  receives  a 
proportion  of  the  increased 
revenue  or  cost  savings) 

On  another  basis  (please  specify) 


11.  For  what  types  of  project  or  project  element  are  each  of  these  pricing  mechanisms 
most  appropriate?  (For  example:  Systems  Design  projects,  Programing,  IT 
consultancy) 

Fixed-price  Project  types   


Basis 


Rating 
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Time-and-materials  Project  types 


Value-based  (ie.  the  SI  Project  types 

vendor  receives  a 
proportion  of  the  increased 
revenue  or  cost  savings) 


On  another  basis  (please  specify) 
  Project  types 


12.  What  do  you  see  as  the  advantages,  if  any,  of  each  of  these  approaches  ? 
Fixed-price 


Time-and-materials 


Value-based  (i.e.  the  SI  vendor  receives  a  proportion  of  the  increased  revenue  or  cost 
savings) 


On  another  basis  (please  specify) 
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13.  Please  rank  your  usage  of  the  different  pricing  mechanisms  for  large 
projects  <  ($500K). 


Price  Mechanism 


Rank 


Fixed-price 

Time-and-materials 

Value-based  (i.e.  the  SI  vendor 
receives  a  proportion  of  the  increased 
revenue  or  cost  savings) 

On  another  basis  (please  specify) 


14.  Please  rank  your  usage  of  the  different  pricing  mechanisms  for  small  projects 

Price  Mechanism  Rank 

Fixed-price   

Time-and-materials   

Value-based  (i.e.  the  SI  vendor  receives   

a  proportion  of  the  increased  revenue  or 
cost  savings) 

On  another  basis  (please  specify)   
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Overall  Attitude 

15.  To  what  extent  do  you  believe  that  : 

(Please  rate  the  following  on  a  scale  of  1-5  where  1  =  strongly  disagree 
and  5  =  strongly  agree) 

Vendors  should  take  more  responsibility  for  project  risk    • 

Vendor  profitability  is  adequate  cover  their  risk   

Clients  should  take  an  equal  share  of  project  risks   

Fixed-price  projects  lead  to  an  adversarial  relationship 

between  client  and  vendor   

Improved  partnerships  would  lead  to  improved  ability  to 

meet  the  clients'  business  need   

Value-based  pricing  would  lead  to  increased  project  success   

Client  and  vendor  should  share  the  risks  and  the 

rewards  of  projects   

A  fixed-project  price  is  essential  for  budgeting  purposes   

Vendor  incentives  should  be  based  on  a  valued-based 

price  component   

Vendors  should  not  be  offered  incentives  beyond  the 

fixed-price  agreed   

16.  Do  you  favour  the  use  of  a  preferred  supplier  for  information  services  projects  ? 
( Y/N) 

Why  ? 
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Vendor  Interview  Guide 


I  am  calling  from  INPUT,  an  international  research  and  consulting  company.  We  are 
conducting  a  study  on  vendors  approaches  to  estimating  and  managing  risk  in  SI 
projects  and  the  associated  impact  on  contracts  between  SI  vendors  and  their 
customers.  The  information  that  you  provide  will  be  confidential  and  neither  your 
name  nor  your  company's  name  will  be  connected  with  any  information  in  the  study. 
In  return  for  your  assistance,  we  will  provide  you  with  a  summary  of  our  findings  at  no 
charge. 

There  are  many  potential  areas  of  risk  and  uncertainty  in  bidding  for  and  performing 
projects.  Examples  include:  estimation  of  costs,  establishment  of  profit  targets, 
definition  of  schedules,  availability  of  skills  etc.  I  would  like  to  start  by  asking  some 
questions  regarding  the  identification  and  management  of  risk. 

1.    On  a  scale  of  1-5,  where  1  =  negligable  and  5  =  very  significant,  how  much  risk 
would  you  say  there  is  in  each  of  the  following  areas  : 

Factor  Rating 

Initial  identification  of  requirements  by  users   

Lack  of  user  involvement  during  course  of  project   

Project  management  by  customer's  project  manager   

Project  management  by  your  project  manager   

Resource  estimating  by  the  customer   

Resource  estimating  by  your  organization   

Risk  evaluation  by  customer  at  start  of  the  project   

Risk  evaluation  by  vendors  at  start  of  the  project   
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Factor 


Rating 


Control  over  end-user  requests  for  change   

Other  (please  specify)    

2.  Is  there  a  level  of  risk  above  which  you  will  not  accept  a  project?  What  is  this 
threshold  ?  (For  example,  if  estimated  profit  is  less  than  25%  of  revenue) 

3.  What  percentage  of  Invitations  to  Tender  (or  Requests  for  Proposal)  that  you 
receive  do  you  decline  because  of  the  risks  involved  (rather  than  because  of  strong 
competition)  ?   % 

4.  What  steps  do  you  take  to  minimize  the  risk  in  projects  which  you  perceive 
comparitively  high  potential  risk? 


5.    How  often  do  you  use  each  of  the  following  pricing  methods  as  a  basis  for  pricing  of 
contracts?  Please,  rate  each  method  on  a  scale  of  1-5  where  1  =  rarely  and  5  =  very 


Fixed  price 

Time  and  materials 

Value-based 

(i.e.,  the  SI  vendor  receives  a  proportion 
of  the  increased  revenue  or  cost  savings) 

On  another  basis  (please  specify) 


6.    What  do  expect  these  ratings  to  be  in  two  years  time  ? 

Basis  Rating 

On  a  fixed  price  

On  a  time  and  materials   

On  a  value-based   


frequently 


Basis 


Rating 
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Basis 

On  another  basis  (please  specify) 


7.  Would  you,  please,  outline  the  approaches  you  use  for  each  of  the  following  tasks, 
identifying  any  models,  tools  or  methodologies  used  ?  What  are  the  key  methods  of 
evaluating  risk  in  each  of  these  areas  ? 

a)  Estimating 

b)  Overall  Project  Management 

c)  Sub-contractor  Management 

d)  Project  Pricing 

8.  Roughly,  what  percentage  of  projects  do  not  satisfy  the  clients  business  needs  as 
defined  at  the  start  of  the  project  ? 

By  number   %   .    By  value   % 

9.  Roughly,  in  what  percentage  of  projects  do  costs  exceed  revenue  ? 
By  number   %       By  value 


% 


10.  Has  consideration  of  risk  reduction  resulted  in  any  of  the  following  becoming 
included  in  your  client  contracts? 

Project  milestones  (  Y  /  N  ) 

Performance  clauses  (  Y  /  N  ) 

Guaranties  or  warranties  (  Y  /  N  ) 
Bonus  payments  for  early  completion/ 

costs  less  than  estimates  etc  (  Y  /  N  ) 

Others  (please  specify)  (  Y  /  N  ) 


Thank  yon  for  your  assistance. 
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